Tag Archives: Open AI

“Open” AI Is Not Open, and “Local” AI Is Not Free

Reading Time: 3 minutes

  I’m a student planning to run a local AI assistant on my own computer. I thought this means a free and open tool: download a model, run it, and forget cloud bills. But “open” may describe only one part of a model and “local” changes where costs appear rather than making them disappear.

The case for local control

  Michael Kerkhoff argues that open-weight models reduce vendor lock-in: if a provider changes prices or rules, a company still has another option. He cites a Red Hat survey in which 63% of organizations used open-weight models, and he suggests moving 10-20% of AI spending to self-hosting. This is worth considering, but Context Studios sells local AI services, and its claim that open models are a “90% solution for 95% of use cases” has no evidence in the article. The strategic argument is plausible and the performance claim needs proof.

Myth 1: “Open” means open

  Open weights are downloadable model parameters. They do not automatically reveal training data, code, methods or full documentation. Liesenfeld and Dingemanse surveyed more than 45 generative AI systems and argue that openness has several dimensions: weights, data, code, documentation and licences. Calling a model open because its weights are available reduces a complex question to one feature.

  Linthicum makes a related point: companies can promote openness while keeping essential parts closed, and even models such as LLama 3 can have restrictive terms. Managers should ask what they may legally change, whether the documentation is complete and where the model can run.

Myth 2: “Local” means free

  Local AI is not automatically cheaper. Pan, G. et al. compare nine open models with six commercial APIs across 54 scenarios. Counting hardware and electricity, a small model on a roughly $2,000 GPU can break even in a few months and medium models need about two years and large ones five years or more. But their model assumes full hardware use and excludes staffing and maintenance.

  SitePoint includes labor and depreciation and gets a different answer. At 500,000 tokens per day, a local setup costs about $6,457 in year one, versus $1,260 for OpenAI’s API and $360 for a hosted open-weight API. At five million tokens per day, local costs about $18,387, still above the API estimates. The two studies do not simply cancel each other out: they count different costs. SitePoint also uses June 2025 prices, and higher European electricity prices push break-even further away.

My view

  This is a classic make-or-buy decision. The real question is not “local or cloud”, but how much control a firm wants to pay for. Total cost of ownership (TCO) includes people, time and risk, not only a GPU. SitePoint’s numbers also show a third option: using an open model through someone else’s API, which was the cheapest in its light and medium scenarios. So “open” does not always mean “local”.

What managers should do

  Decide by workload and team skills, not by labels or enthusiasm. Cloud APIs may suit light or irregular use, while local deployment may be justified for high volumes or strict data-residency needs and  a hybrid setup can keep sensitive or predictable tasks local and use cloud models when needed. Before choosing, ask: What workload are we actually running? What data must stay under our control? Who will maintain and secure the system?

AI Disclosure

Engine used: ChatGPT
Prompts:

  • “Find me 5 recent articles about running AI locally on your own computer and about open AI models for companies. I want some articles that disagree with each other. For each one, give the title, author, date, link, and one sentence about what it says.”

  • “Compare these 5 articles and tell me where they disagree or don’t agree on something. Explain it in simple English.”

  • “Give me 5 more ideas about local AI that people could argue about. I want to write a blog post about it for my management class.”

References

Tagged ,