The model landscape: what to use, when, and who owns it

Hosted APIs (OpenAI, Anthropic, Google, xAI, and others) sell you tokens over HTTPS, with a model name that will change. You do not own the weights. You own a contract: data handling, region, uptime, rate limits, price. That is not a lesser purchase. For most thin slices it is the right purchase. You are buying a completion interface with a legal wrapper. If you catch yourself saying 'we are an OpenAI shop' as an identity, you have already started the lock-in. You are a shop that calls an interface. The interface can move.

Open weights (Llama, Mistral, Qwen, and cousins) mean you can run the model on your iron or a specialist host. You own more of the stack and all of the ops. 'Open' is not 'free,' and it is not automatically safer — you still have to secure the box, the prompt logs, and the plugins. Open is a control and an ops bill. It is the right call when data residency, air-gap, or unit cost at huge volume demand it, and when you have a team that can actually run GPUs. It is a daydream when a security lead says 'open source' as a way to say no to a US vendor without staffing a platform team.

Enterprise platforms (Bedrock, Azure OpenAI, Vertex, Databricks Mosaic, various AI gateways) sell you a control plane: SSO, VPC, logging, multiple models behind one key. Delivery managers should default to 'platform + eval,' not 'one model forever.' The platform is a door. Behind the door are model names that will churn. Your prompts, evals, and data should survive a swap of what is behind the door. If your code hard-codes a model string in twelve services, you did not buy a platform. You bought a snowflake with SSO.

What you are not buying: a brain that knows your company, a permanent name, or a strategy. Vendors will talk as if the model is the product. For a delivery lead the product is the path: retrieve, generate, eval, log, human. The model is a component with a price and a deprecation date. Treat it like you treat a middleware vendor — important, replaceable, contractual. The teams that lose this plot spend a year in 'model selection' and never write an eval set.

Hosted vs open is a trade, not a religion. Hosted wins on time-to-slice, quality at the frontier, and not running GPUs. Open wins on data gravity, unit cost at scale, and not sending tokens off-box. Hybrid is common and adult: hosted frontier for the copilot that needs language quality, small open or small hosted model for classification at 80k tickets a night. You can say that in a steering meeting without picking a tribe. If the room wants a tribe, you are in a brand meeting, not a design meeting. End the brand meeting.

Write the default sentence and keep it until evals move you: 'We will pick a hosted frontier model behind our enterprise gateway for the thin slice, measure quality and cost, and keep the interface swappable. We will not fine-tune until retrieval and evals are boring.' That sentence kills three stalls: the favorite-model stall, the fine-tune stall, and the open-source-as-identity stall. You can still choose open weights later. You will choose it with numbers, not with a blog. Put the sentence on the charter. When the room relapses into brands, read it aloud. A charter sentence beats a mood. Moods change at every keynote. Charters change when the eval table moves.

Frontier general models (the GPT / Claude / Gemini / Grok class) are the right default for language-heavy work: drafting, extraction, reasoning over a retrieved packet, tool use. They differ in tone, refusal style, long-context behavior, and cost — you find out with evals, not Twitter. A leaderboard is a public exam. Your task is a private exam. The model that wins MMLU may lose on your RAID miner because it is too chatty, or it refuses too often, or it cannot keep a JSON schema. Run your exam.

Small / cheap models are for classification, routing, and high-volume extraction once a frontier model has proven the task. Starting with the biggest model is a valid discovery move. Staying there in production is often a cost bug. A worked pattern: use a frontier model to label 500 tickets and to write the rubric. Then train or prompt a small model to apply the rubric at 80k a night. Eval the small model against the frontier labels. If it holds, you just cut the bill by a large integer. If it does not hold, you stayed on frontier with evidence. Either outcome is adult.

Embedding models are a different product: they turn text into vectors for search. Do not use a chat model as your search index. Speech, image, and video models are extra interfaces; most enterprise value in year one is still text plus retrieval. When a sponsor says 'we need multimodal,' ask which input is actually arriving. If the input is PDFs of text, you need a parser, not a vision strategy. If the input is photos of damaged shipments, then you need a vision model and a labeled set. Multimodal is a data type, not a maturity badge.

Reasoning / 'thinking' models are frontier models that spend more tokens (and time) on internal scratch before answering. They can help on hard multi-step tasks. They are expensive and slow for 'extract the due date.' Do not put a reasoning model on a high-volume classification path because a demo looked clever. Route: cheap model for easy, frontier for medium, reasoning for the 4 percent that fail a confidence gate. Routing is delivery architecture. A single 'smartest' model is a demo architecture.

The taxonomy you need on a slide is four boxes, not twenty logos. (1) Frontier hosted chat/completion. (2) Small / cheap classifiers and extractors. (3) Embeddings. (4) Specialist (speech, image, code, rerankers). Pick one box per step of the path. A RAG copilot is usually embeddings plus frontier completion. A ticket router is a small classifier. An after-hours voice tree is speech-to-text plus a small router plus a frontier drafter. If your architecture diagram is a pile of logos, you do not have a taxonomy. You have a shopping list.

When a smaller/cheaper model wins: the task is narrow, the eval set is good, latency or cost is the constraint, and the failure cost is low enough to retry or to fall back. When it loses: the task is open-ended, the user can see the seams, tool use is messy, or the eval is not built yet. Discovery with a big model, production with the cheapest model that holds the eval, fallback to the big one on low confidence. Write that as a policy. It will save you from both stingy quality and theatrical spend.

The question that kills deals is not 'which model is smartest.' It is: where does the prompt go, is it retained, is it used for training, which region, who can see logs, and can we run in our VPC. A slightly worse model that legal will sign is a better delivery choice than a better model stuck in procurement for six months. You have watched this movie with every SaaS. AI is the same movie with a more alarming data class, because prompts contain the documents people should not have pasted.

Zero-retention APIs, private regional endpoints, and 'we already have a BAA / DPA with this cloud' are delivery facts. Collect them in week one of any engagement, not at go-live. If the company already has a signed DPA with Azure and nothing with a new model vendor, Azure OpenAI as a door is not a technical preference. It is a calendar preference. You can still eval Claude or Llama behind other doors later. First slice uses the door that opens.

Residency is a path, not a logo. User types in a browser. The app server sees the prompt. Logs may store it. The gateway may store it. The model vendor may store it for abuse monitoring even on a 'we don't train on your data' plan. A support engineer at the vendor may see a flagged prompt. Your own admin may search the prompt store. Draw copies. Security partners will not sign a slide that says 'data stays in the tenant' if you cannot show the copies. They will sign a diagram with honest boxes.

Consumer ChatGPT (or any consumer copilot) is not an enterprise data path, even if an executive uses it every morning. No DPA you control, training defaults you do not like, paste culture, no audit of what left. Your operating note on day 7 will say this. Say it today too. The failure mode is a well-meaning PM pasting a customer dump into a consumer tool 'just to see.' That is an incident, not a learning moment. Give them an approved path or they will use the unapproved one.

Worked example. Regulated firm, customer-complaint extraction. Data is sensitive. Legal will not send raw complaints to a US-only consumer API. Options: (a) Azure OpenAI or Bedrock in a named region with no-training and retention terms in writing. (b) Open weights in the VPC if the platform team actually exists. (c) Do not do the use case yet. Option c is allowed. A delivery lead who cannot say 'not this quarter' will force option a through a six-month review and call the delay 'IT.' The delay was an unplanned data class. Plan the class.

Put the path on the charter as a one-line: 'Prompts go from the internal app to Gateway X in region Y, vendor Z, retention = 0 days, training = no, logs retained 30 days in our SIEM, no consumer tools.' If you cannot fill those blanks, you do not have a vendor. You have a demo. Fill them before you argue about temperature. A steering committee can live with a named gap ('region TBD, legal due Friday'). They cannot live with a smile and 'the enterprise version handles that.' Named gaps have owners. Smiles do not.

1) What is the interface we own — our prompts, evals, and data — vs the model name we can replace? If the answer is 'our code calls gpt-something-dated in twelve places and the prompts are in a chat history,' you are already locked. Own a gateway, a prompt store with versions, and an eval harness that takes a model name as a parameter. That is the interface. The model string is a config. Config you can change in a week is freedom. A string copy-pasted into twelve services is a program of work disguised as a preference.

2) What is the cost at 10× volume of the POC? Demo cost is a lie — day 5 will do the math. Ask it here anyway, because vendors will price the POC in charm. Charm does not survive 2,000 users. If they cannot help you envelope 10×, they are not a production vendor yet. You can still use them for discovery. Do not sign a platform paper on discovery prices. A platform paper on discovery prices is how you explain a surprise invoice to finance six months later. Put 10× in the paper or do not sign it.

3) What is the data retention and region story in writing? Not in a blog, not in a sales deck, in the DPA and the product docs that match the SKU you are buying. 'We don't train on your data' can still mean 'we retain for 30 days for abuse.' Those are different sentences. Write both on the charter. If the region is only US and your policy is EU, stop. Do not let a champion say 'we'll get EU later.' Later is a second procurement.

4) What happens when the model version is deprecated (they all are)? You need a pin, a notice window, and an eval you can re-run on the successor in a week, not a quarter. Teams that skip this wake up on a Monday to a different tone, a different refusal style, and a broken JSON schema. Deprecation is a RAID item with an owner. The owner is not 'engineering sometime.' The owner is the person who can re-run the eval set.

5) What is the eval score on OUR tasks, not a public leaderboard? If a vendor cannot answer 5, they are selling a demo. If your own team cannot answer 1 and 4, you are building a snowflake. Bring a 20-row eval set to the vendor bake-off. Score format, groundedness, and task success. The winner is the one that wins your set at a cost and latency you can live with, on a path legal will sign. That is the whole method. Everything else is fandom.

Write the five on a card. Reuse them on day 19 and in every vendor meeting this year. You will feel repetitive. Repetition is the job. Vendors are trained to move you off these questions and onto a roadmap slide. Smile, and ask number 5 again. If you need a sixth: 'Show me the kill / switch. How do we leave.' If they cannot describe an export of your prompts, evals, and logs, you are not a customer. You are a resident.

Azure OpenAI is OpenAI models behind Microsoft's enterprise door: Entra ID, regional endpoints, the DPA you may already have, the landing zone your cloud team already fights about. You are not 'using Microsoft's model' in the research sense. You are using a hosted model through a control plane your company might already know how to approve. That is often the entire reason it wins. Quality is the second conversation. The first is: we can turn this on without a new legal entity.

Bedrock is Amazon's door: multiple model vendors (Anthropic, Amazon's own, others) behind IAM, VPC endpoints, and a AWS-shaped bill. The point of Bedrock for a delivery lead is not 'AWS has AI.' It is 'we can swap model families without swapping the IAM story.' If your company is already a AWS shop with a mature account structure, Bedrock is a calendar choice. If your company is an Azure shop, Bedrock is a second landing zone. Second landing zones are real cost. Count them.

Vertex is Google's door: Gemini and a pile of ML tooling, IAM, VPC-SC, the Workspace gravity well. If the company lives in Google Workspace and the security team already knows Vertex, you have a door. If someone wants Vertex because they like Gemini's demo, that is a model preference wearing a platform hat. Separate them. You can often get Gemini through more than one door, and you can get other models through Vertex. Doors and models are two decisions. Write them on two lines of the steering pack or you will spend a month arguing a hybrid of both. The door is the calendar. The model is the eval. Do not let one vote for the other.

Gateways (LiteLLM, cloud-native gateways, internal API layers) are how you stop hard-coding a door. The app talks to the gateway. The gateway talks to Azure OpenAI this quarter and to Bedrock next quarter if evals say so. Logs, keys, and policy live at the gateway. This is not extra bureaucracy. This is the same pattern you already use for payment providers and email senders. If a builder says the gateway will slow them down, give them a week of direct access for the POC and a requirement to be behind the gateway for anything with more than twelve users.

Lock-in questions specific to doors. Can we bring another model family through this door without a new procurement? What is the network path (public, private endpoint, PrivateLink)? Who can see content in the provider's abuse queue? What is the quota story when the POC becomes 2,000 users on a Monday? What is the price of the door itself (gateway, logging, reserved capacity) vs the tokens? Reserved capacity can be a saving or a trap. If you reserve for a POC volume, you reserved a demo. Reserve after the 10× envelope, or do not reserve.

Do not fight a door out of fandom. If legal will sign Azure OpenAI in three weeks and a direct Anthropic contract in four months, you start on Azure OpenAI even if you prefer Claude's writing. You can eval Claude through Bedrock or a gateway later. The thin slice ships through the open door. You already know this from every other vendor decision. Apply it. The new part is that the people in the room have favorite chat products at home. Acknowledge the favorite. Then pick the door.

A bake-off without an eval set is a taste test. Taste tests hire favorites. Eval sets hire components. You need twenty to fifty rows of real (or realistic, sanitized) inputs, a written rubric, and a score: format match, evidence discipline, task success, latency, cost. Run two or three candidate models on the same rows, same prompt, same retrieval. Look at the table. The table will often surprise the people who had a favorite. That surprise is the point. If nobody is surprised, you probably only ran happy paths. Add the empty-source row and the paint-us-green row and run it again. A bake-off that cannot embarrass a favorite is a demo with extra chairs.

What to put in the fifty rows. Happy paths. Thin sources. Empty sources. Hostile 'paint us green' users. A document that says 'ignore previous instructions.' A number that must not be rounded. A name that must not be invented. The tail class that actually matters (the one refund reason that creates chargebacks). If your eval is only happy paths, every frontier model looks fine and you will pick on vibes. Vibes are how you ship a sycophant. Spend the extra hour writing the hostile rows. They are the ones that will hit production on a Thursday afternoon, and they are the ones a vendor demo will never volunteer.

Score like a delivery person, not like a researcher. You do not need BLEU. You need: would I send this to the steering committee (yes/no), did it invent an owner (yes/no), did it cite a real quote (yes/no), did it return valid JSON (yes/no), p95 latency, tokens, dollars. A spreadsheet is enough for a thin slice. Week 3 will make this grown-up. Today, refuse to pick a model without at least a spreadsheet. If the vendor wants to 'just demo,' let them demo on your rows, not theirs.

Re-run on deprecation, on prompt change, on corpus change. An eval that ran once is a souvenir. Put the re-run on a calendar: monthly, and on every model-name change. The owner is named. This is change control. You already do it for payroll files. Do it for the thing that writes to executives. When the score drops, you have a RAID item with evidence, not a vibe that 'it got worse after the upgrade.' A souvenir in a slide deck will not help you on that Monday. A dated table will.

When the cheaper model wins the table, you have a political problem, not a technical one. Someone's favorite lost. Show the table. Show the 10× cost. Offer a fallback: route the 10 percent failures to the expensive model. That compromise ships. A demand to use the expensive model 'because it's the best' on a task the table says is solved is a brand demand. You can lose that fight once. Do not lose it every time. The eval is how you get to say no without sounding like a cheapskate.

Tie this back to the seats. An AI PM owns the rubric and the kill criteria. An FDE runs the bake-off in the customer environment. A delivery lead puts the table in the steering pack and refuses to pick without it. A transformation lead uses the same method across use cases so the company does not have twelve favorites. Same artifact, different altitude. If you cannot name who owns the eval set, you do not have a model strategy. You have a subscription.

A smaller model wins when the task is narrow, the eval set is good, latency or cost is the constraint, and the failure cost is low enough to retry or to fall back. Classification into twelve tags. Routing a ticket. Extracting a due date into a schema. Those are pattern jobs. A frontier model can do them. So can a cheap one, once you have labeled a few hundred rows with the frontier model and written the rubric. Staying on the frontier for 80k tickets a night is often a cost bug with a quality alibi. The alibi dies when the table says the cheap model holds.

Worked night. 80,000 tickets, 12 tags, one pass. Frontier at notional $3 / $15 per million, 800 input / 60 output per ticket: input 64M, output 4.8M → ~$192 + $72 ≈ $264/night, ~$5.8k/month. Small model at a tenth of that on input and a fifth on output: ~$40/night, ~$880/month. If the eval says tag accuracy holds within two points of frontier, you just found $5k/month and a lot of latency. If the tail class (the one tag that creates chargebacks) drops from 94 to 71, the cheap model lost. The tail class is why you do not pick on average accuracy. Score the class that has blast radius.

Routing is how you get both. Cheap model first. If confidence is low, or the cheap model disagrees with a keyword gate, send that 5–15 percent to frontier. The user still sees one answer. The bill looks like cheap plus a frontier tail, not like frontier on everything. Write the route as policy: 'easy to small, hard to frontier, write-tools never to a model that cannot keep a schema.' Routing is delivery architecture. A single 'smartest' model is a demo architecture. Demos do not have 80k nights. You will.

When the cheaper model loses, say so without theatre. Open-ended drafting the user will read. Messy tool use. A task with no eval yet. A user-facing voice where seams are obvious. Discovery still starts on frontier: you need the good labels and the rubric. Production is allowed to stay there if the table says so. The failure is not 'we used a smart model.' The failure is 'we used a smart model on a pattern job because nobody ran the cheap one on the same rows.' That sentence is a process miss. Put the bake-off on the calendar before the bill becomes a RAID item.

The political problem is real. Someone's favorite lost the table. Show the table. Show the 10× cost. Offer the fallback route so they are not 'giving up quality' — they are paying frontier where it still earns its keep. That compromise ships. A demand to use the expensive model 'because it's the best' on a task the table says is solved is a brand demand. You can lose that fight once if the relationship needs it. Do not lose it every time. The eval is how you say no without sounding like a cheapskate. Cheapskate is a personality. A table is a decision.

Open weights as the cheap model is a separate fork of the same idea. If the 80k night cannot leave the VPC, a small open model on your iron can be the classifier, with a hosted frontier behind a door for the copilot that still needs language quality. Hybrid is adult. 'Everything must be open' and 'everything must be frontier' are tribes. You already ended the brand meeting. End this one the same way: task, eval, data path, cost at volume. If ops cannot run GPUs, the open-weights cheap model is a daydream and you should say so before anyone orders the cards.

Tie it to the default sentence from this morning. Hosted frontier behind the gateway for the thin slice. Measure. Keep the interface swappable. Then, when the slice is boring, run the cheaper candidate on the same eval. If it holds, switch the production string and keep frontier as fallback. If it does not hold, you stayed with evidence. Either outcome is a model strategy. A subscription with a favorite name is not. Write the switch condition on the charter today, even if you will not switch this quarter. A named switch is how deprecation and price-card changes become a week of work instead of a panic.