How to Choose Between Open and Closed AI Models
How to choose between open and closed AI models: six criteria, a simple cost formula and a test you can run on your own examples before you commit.
Every AI product sits on a model, and you have to pick one. The first big fork is whether to use a closed model through someone else's API or an open model you can run yourself. This guide shows you how to choose between open and closed AI models without guessing. You will get six criteria, a cost formula, and a short process you can finish in an afternoon.
If you are new to the ideas underneath, AI Entrepreneur is a free web app of short lessons on how modern AI works and how to build a business with it. Its Intermediate course on the AI product stack has a lesson on this exact choice.
Open vs closed AI models: what the difference actually means
A closed model lives on the provider's servers. You send a request to an API, you get text back, and you pay for what you use. You never see the model's internals.
An open model is one whose weights you can download. In practice "open" usually means open weights, not always open training data, and the license terms differ from model to model. Read the license before you build on one. You can run an open model on your own hardware, rent a server for it, or use a company that hosts it for you.
Both kinds are language models, so they share the same limits. If that idea is fuzzy, read how large language models work first.
Why this choice matters for an AI business
The model is likely your largest ongoing cost and your largest dependency at the same time. It affects what you can promise customers, what your margins look like, and how much a provider's decision can hurt you.
Run the numbers early. Every request to a model has a real cost, and your model choice sets it. That is one reason AI company economics differ from SaaS.
It also affects how fast you can move. A closed API is usually the quickest way to a working prototype. An open model can give you more control later. You can start on one and revisit the choice once you have real usage.
The six criteria that should drive your decision
Do not ask "which kind is better?" Ask how each kind scores on your product. Use these six criteria:
- Quality on your task. How well does the model do your specific job, not a general benchmark?
- Cost at your volume. What do you pay per request today, and what will you pay at ten times the traffic?
- Privacy and data rules. Are you allowed to send this data to a third party?
- Control. Do you need to adjust the model, fix its version, or run it somewhere specific?
- Lock-in. How hard is it to switch if a price, policy or model changes?
- Effort to run. Who on your team will operate it, monitor it and fix it at 2 a.m.?
Weight these for your product. A small team with no operations experience should weigh criterion six heavily. A product that handles sensitive records should weigh criterion three heavily.
Cost: API pricing versus self-hosting at scale
With a closed API, your cost scales with use. The basic formula is:
monthly API cost = requests per month × tokens per request × price per token
Providers usually price the tokens you send and the tokens you get back differently, and prices change, so look up the current price for any model you consider. The lesson "Model APIs, Tokens and What a Request Costs" is about this.
With self-hosting, the cost is mostly fixed:
monthly self-hosting cost = server rental + engineering time + monitoring and upkeep
The server costs money whether it is busy or idle, so there is a break-even point. Below a certain volume, paying per request is cheaper. Above it, a fixed cost spread over many requests can win.
Here is how to find your own break-even point:
- Count your real requests per month, or your best estimate.
- Measure tokens per request on a sample of 50 real inputs.
- Compute the API bill with the formula above.
- Get a quote for the server you would need, and add honest time for someone to run it.
- Compare the two numbers at your volume and at ten times your volume.
Engineering time is the cost people leave out, and it can decide the answer for a small team.
Quality, privacy and control: where each type wins
Quality
Do not assume either side is better. Ask whether a model is good enough for your task at a price you can afford. A smaller model on a narrow job can beat a larger one on cost and speed.
You cannot settle quality by reading about it. Build a small set of real inputs with known good answers and run every candidate on it. We cover this in how to build an eval set for an AI feature.
Privacy
If you run an open model on infrastructure you control, your data does not have to leave it. That can matter a lot for health, legal or financial data, or for customer contracts that limit who may see information. A closed provider may offer data terms that satisfy your needs, so read them. Ask a lawyer when the data is sensitive, because this post is not legal advice.
Control
With an open model you can pin an exact version so it never changes under you, and choose where it runs. With a closed API, the provider decides when versions change or retire. That is manageable if you test regularly, but you do not hold the controls.
Lock-in and platform risk: how to keep your options open
The real danger is not choosing closed. It is building so tightly around one provider that you cannot leave. A price change, a policy change or a retired model then becomes your emergency. We go deeper in platform risk and building a business on someone else's model.
You can keep your options open with a few habits:
- Wrap the model call. Put every model request behind one function in your code. Changing providers then means changing one place.
- Keep your prompts in files. Store them apart from the code.
- Keep your eval set. It is the fastest way to find out whether a new model is good enough to switch to.
- Own your data. Keep your documents and examples in your own storage, not only inside a vendor's tool.
A business can survive a model swap if the value sits in your product, your data and your customers. If your whole product is the provider's model with a thin layer on top, you have a bigger problem than open versus closed.
A simple decision framework: how to choose between open and closed AI models
Here is a process you can run today. Imagine you are building a tool that reads incoming support emails and sorts each one into a category with a draft reply.
- Write the job in one sentence. "Read a support email and return a category and a draft reply." A clear job makes every later step easier.
- List your hard constraints. Can the emails leave your servers? Must the model run in a certain place? If a constraint rules a type out, stop there.
- Build the eval set. Collect 30 to 50 real emails and write the right category for each. Add a note on what a good reply looks like.
- Start with the easiest option. For most small teams that is a closed API. Run your eval set and record the score.
- Try one open model on the same set. Use a hosted version first so you test quality without running servers. Compare scores.
- Price both at your volume. Use the formulas above. Include engineering time on the self-hosting side.
- Pick the one that passes the quality bar at the lowest total cost and risk. Write down why, and when you will check again.
Use a prompt like this to run the same test on each model:
You are a support triage assistant. Read the email below. Return the category (billing, bug, how-to, other) on the first line and a two sentence draft reply on the second. Email: [paste email]
Keep the prompt identical across models so the comparison is fair. To turn one good prompt into a working tool, read how to build a simple AI tool from one good prompt. Revisit the decision when your volume, your data rules or the available models change.
Frequently asked questions
Is an open model always cheaper than a closed one?
No. Open weights can be downloaded for free under their license, but running them costs servers and engineering time. At low volume, paying per request is often cheaper.
Should a beginner start with an open or a closed model?
Most beginners get to a working prototype fastest with a closed API, because there is nothing to set up. Keep your model calls in one place so you can test an open model later.
Can I switch models later?
Yes, if you plan for it. Wrap the model call, keep prompts in separate files and keep an eval set so you can check any new model quickly.
How do I know if a model is good enough?
Test it on real examples from your own task, not on general benchmarks. A small eval set with known good answers gives you a score you can compare across models.
Get started
The lesson "Choosing Models: Big, Small, Open, Closed" is in the Intermediate course The AI Product Stack, next to lessons on model APIs and retrieval. Lessons take 8 to 12 minutes each.
Every lesson is free. Start learning with AI Entrepreneur: sign in on the home page and take your first short lesson today.
