An AI Primer
for Product Leaders

LLMs as Artifacts You're Shipping

The model you call through an API is not "the model." It's a specific, opinionated artifact that came out of a long pipeline of choices — a base model, then some supervised fine-tuning, then preference training, then a refusal regime, then maybe a thinking mode, then a final quantization pass. The thing on the other end of your HTTPS request is more like a curated build than a law of physics. If you treat it like a black box you'll be surprised by it forever. If you treat it like a shipped artifact with a changelog, you can actually reason about it.

Base vs post-trained

A base model is what falls out of pretraining: a fluent next-token continuer that has read most of the internet and has no idea it's supposed to be helpful. Hand it a question and it might respond with a list of similar questions, because that's the shape of FAQ pages it saw during training. The base model is the world-model. Almost no one ships it.

What you ship is the post-trained model — base plus SFT plus some preference-learning stack (RLHF, DPO, RLAIF, RLVR, whatever this quarter's recipe is). Post-training is where the model learns to be an assistant rather than an autocomplete. It's also where every label you actually care about as a PM — refusal patterns, formatting defaults, tone, "personality," willingness to use tools, how it handles uncertainty — gets stamped in.

"Instruct-tuned" is a loaded term. It used to just mean "we did SFT on instruction-following data." Today it bundles a dozen training choices a lab made about who the model should be. When Anthropic and OpenAI ship different "instruct" variants of comparable base capability, the differences you feel as a user are almost entirely post-training differences. Same engine, different driver education.

Personality is a training artifact

Claude's slightly-anxious helpfulness, GPT-5's terse-confident defaults, Gemini's bureaucratic-when-cornered tendency — these aren't accidents. They are the cumulative residue of which examples got labeled "preferred" during preference training, which refusal patterns got rewarded, and which "thoughtful" responses got reinforced when raters were tired. The model's vibe is a training byproduct of the lab's taste.

This matters because you cannot fully prompt your way out of it. You can nudge a model toward a voice, but the gravitational pull toward its trained default is real, and it gets stronger under load (longer contexts, harder tasks, ambiguous inputs). If your product needs a specific voice and the model's natural voice fights you, that's a model-selection signal, not a system-prompt problem.

Mode collapse and refusal training, from a product POV

Preference training has a known failure mode: mode collapse. If raters consistently prefer a certain shape of response — bulleted, hedged, "I'd be happy to help" — the model learns to produce that shape for everything, including tasks where it's wrong. The famous "as a large language model, I cannot..." era was mode collapse on refusals. The current "let me break this down into a few parts" reflex is mode collapse on structure. Your users feel it as the model being weirdly formal or oddly verbose. It's not a bug per se; it's a side effect of how preference data gets collected at scale.

Refusal training is similar. Labs RL the model to refuse a long list of harmful requests. Done well, that's safety. Done at volume by tired contractors, the refusal classifier overfires — your model declines to translate a Wikipedia paragraph because it mentioned a chemical, or hedges on a recipe because it involves a knife. As a product person, you should expect a few percent of your traffic to hit a false-positive refusal, and you should design the UX to surface it as "the model declined this; tell us if it shouldn't have" rather than letting it look like a server error.

Why temperature isn't the knob you want

Temperature is genuinely useful when you're picking between determinism (extraction, classification) and exploration (creative drafts). But for most product decisions it's the wrong dial. The actual variance in your output usually comes from the prompt, the system message, the model version, and — increasingly — whether you're using a reasoning model or not. Sweeping temperature is the AI equivalent of tuning JVM flags before you've profiled your code.

What you usually want: stronger structure (JSON schemas, structured outputs), better grounding (retrieved context with citations), or a different model entirely. If you find yourself fiddling with temperature to chase quality, you've probably misidentified the bottleneck.

The bleeding edge: model surgery

For most of the last few years, the model was an opaque box you could only steer from the outside (prompt, tools, retrieval). That's changing. Sparse autoencoders, feature steering, activation patching — the broad mech interp program — have produced the first credible methods for reaching inside a model and adjusting specific learned concepts. Anthropic's "Mapping the Mind of a Large Language Model" work is the cleanest public artifact here.

It's not yet a product surface. But it's plausibly where the next layer of customization comes from — "tune down sycophancy on this customer's deployment," "boost legal-domain features for this tenant." If your three-year product roadmap assumes the model is a black box forever, revisit it.

How Frontier Models Actually Get Built

If you internalized one mental model in 2022, it was probably "more compute, more data, smarter model." That model is now wrong, or at least leading. Frontier capability in 2026 comes from somewhere else, and the somewhere-else is where every interesting fight in AI right now is happening. Worth understanding even if you never train anything yourself, because it determines what your suppliers can charge and how fast the ground will move under your product.

Scaling laws and the data wall

The Kaplan scaling laws said you could predict model loss from parameters, data, and compute, and that they all needed to grow together. The Chinchilla paper corrected the data-vs-parameters ratio — Kaplan-era models were under-trained on data — and gave the industry a clean recipe for "Chinchilla-optimal" training. For a while, the path forward looked like a treadmill: scale the model, scale the data, scale the GPUs.

That path hit two walls roughly in tandem. First, the data wall: frontier labs had ingested most of the high-quality public text on the open web. There was no obvious "10x more tokens" to feed the next generation. Second, returns from raw scaling started to bend. Doubling pretraining compute did not double anything you could measure on the eval suite. The "scaling has hit a wall" headlines around the GPT-4-to-GPT-5 era were oversold — scaling isn't dead — but they were directionally right about pretraining.

Pretraining is becoming commodity

This is the load-bearing claim, and it's contested, so let me be careful. The strong version: base models in 2026 are converging in capability. DeepSeek's open-weights V3-class bases, Llama, Qwen, the closed-lab equivalents — they are all roughly in the same neighborhood as pure language models. The differentiation has moved downstream into post-training, RL, and inference stacks.

The counter-argument, which the big labs would make: pretraining isn't commodity, it's just that the next leap requires a different recipe. The "synthetic data + RL is the new pretraining" framing argues that you can keep scaling capability by generating training data with current strong models and reinforcement-learning the next generation on it, especially when the data is verifiable. There's a sneaky implication in there: the labs with the strongest current models have a flywheel for the next ones.

Both can be true. The base-model commodity story explains why DeepSeek can match closed labs on language capability at a fraction of the training spend. The synthetic-RL flywheel explains why frontier reasoning still tilts toward whoever has the best current reasoners. As a PM, what matters: the gap on "general LLM" is shrinking, the gap on "reasoning at the frontier" is real and persistent.

RLHF → RLAIF → RLVR

The post-training story is best told as an evolution of where the reward signal comes from. RLHF uses humans to compare pairs of model outputs and trains a reward model on those preferences. It works, but human labeling is slow, expensive, and noisy. RLAIF replaces (some of) the human labelers with a strong AI model. Cheaper, more consistent, scales further; the cost is that you're training the next model to match the previous model's taste, which has obvious drift risks.

RLVR — RL from verifiable rewards — is the move that unlocked the current reasoning era. Instead of asking "which answer is better," you ask "did the code compile and pass the tests? did the math proof check? did the chain end in the right boxed answer?" When the reward is verifiable, you can generate millions of training rollouts overnight and grade them automatically. The reward is dense, objective, and basically infinite.

This is what's happening when a frontier lab "trains a reasoning model." You sample long chains of thought, you check the final answer against a verifier, you reinforce the chains that won. The model isn't learning to "think" in a metaphysical sense — it's learning that producing certain intermediate-token patterns leads to more reward when those patterns lead to verifiable correct outputs. The "thinking" is rewarded behavior, not magic.

Why DeepSeek-R1 mattered

DeepSeek's R1 paper was the watershed because it showed, in public, with open weights, that you could get o1-level reasoning out of an open-base model with a relatively crisp RL recipe. The result mattered along three axes. Capability: open weights were suddenly competitive with the closed frontier on reasoning, not just chat. Cost: it dropped the floor on what reasoning had to cost. Method legibility: the recipe was public enough that the rest of the field could reproduce it within months, which is why you saw the o3/Gemini-Thinking/Claude-extended-thinking wave land in roughly the same year.

The downstream effect on pricing was immediate. Reasoning models that used to cost $60/Mtok started showing up at a tenth of that. Your supplier negotiations got easier; the labs' margins got tighter.

Distillation, briefly

You can train a small model to imitate a big one's outputs. That's distillation, and it works shockingly well for narrow tasks — sometimes you get a smaller model that matches the big one on the distribution you trained for, at 10x lower inference cost. What distillation does not give you: out-of-distribution robustness, novel reasoning, or anything resembling the teacher's full capability surface. The student is a specialist that mimics the teacher's behavior on the training set, not a junior version of the teacher.

For products, distillation is mostly a cost-optimization story — take a strong teacher (Claude Opus, GPT-5) and distill its behavior on your task into a much cheaper model you can serve at volume. It is not, despite the marketing, "frontier intelligence at 1/10th the price."

MoE as an inference-cost story

Mixture-of-experts architectures route each token through a small subset of the model's parameters. The headline number ("a 1T-parameter MoE!") is mostly marketing — only a fraction of those parameters fire on any given token. The real story is that MoE lets you stuff a lot of specialized capacity into a model whose inference cost looks much smaller than its parameter count. It's primarily an inference-cost lever, not a quality lever. Treat it as such when you're reading model cards.

Inference Economics & the Triangle

Inference economics is where AI product strategy actually gets decided. Cost, latency, and quality form a triangle, and every meaningful product choice — model selection, caching, batching, reasoning mode on/off, context length — is a trade between two of the three. The teams that win in this market are the ones who internalize the triangle as a moving constraint, not a static menu.

The KV cache, and why prompt caching is a product enabler

Inside the model, attention is computed against keys and values for every prior token. The KV cache is the bookkeeping that holds those intermediate states so the model doesn't recompute them per generated token. It's why generating the 100th output token is roughly as cheap as generating the second — most of the work was done up front.

Prompt caching is the API surface on top of this. You mark a chunk of your prompt as cacheable. On subsequent calls that include the same cached prefix, the provider serves the precomputed KV state instead of re-attending. You pay a small premium on the first call to "warm" the cache, and ~10% of input price thereafter. Latency to first token also drops noticeably.

The eviction policy is the bit nobody reads carefully but everybody trips over. Anthropic's cache has time-based TTLs (5 minutes default, up to an hour); OpenAI's is best-effort with shorter eviction; Google's is its own thing. Your product's session shape determines whether prompt caching is a 5x win or a no-op. A long-running coding agent re-hits the cache constantly. A bursty consumer chatbot where users come back hours later doesn't. Design around the eviction policy or pay full price every cold start.

Batch APIs vs realtime

Every major lab now ships a batch API at roughly 50% off list. The deal: submit a pile of requests, get results back within 24 hours. For anything that's not user-facing — backfills, eval runs, document classification, nightly enrichment — you should default to batch. Half the teams I talk to are running RAG ingestion through realtime endpoints at full price because nobody flipped the switch.

Streaming is a UX trick

Streaming doesn't reduce total latency. It reduces perceived latency by letting the user start reading after the first 100-200ms (TTFT) instead of after the full 8-second generation. Same total time, vastly better feel. The interesting product question is when not to stream: structured outputs where the user can't act on a partial response, agent steps where the model needs to decide before any visible output, anything that gets post-processed before display. Streaming for the sake of it adds engineering surface without UX benefit.

Speculative decoding

Worth knowing the name. A small "draft" model proposes the next few tokens, the big model verifies them in parallel, and you commit the prefix they agree on. You get N tokens for the cost of slightly more than one forward pass when the small model is correct, which it usually is on easy spans. This is most of why frontier APIs feel faster in 2026 than they did in 2024 even on similar model sizes. You don't configure it; it just lives in the inference stack.

Long context isn't free RAM

Vanilla attention is O(n²) in sequence length, which used to make long context a quality and cost disaster. Modern inference stacks (FlashAttention, paged attention, sliding-window tricks) have mostly tamed the compute cost — you don't pay quadratic dollars anymore. But you still pay:

Quality: "lost in the middle" is alive and well. Models attend disproportionately to the start and end of long contexts; relevant info in the middle gets dimmed. Real-world recall on 200k-token contexts is materially worse than on 20k.

Cost: per-token pricing is linear, but 500k tokens is still 500k tokens. Prompt caching helps when the chunk is reusable; it doesn't if you keep editing it.

Latency: TTFT scales with prefill, which scales with input length. A 500k input doesn't cost quadratic time anymore, but the user waits seconds more for the first token. For a chat product, that's a UX killer.

The right mental model: long context is a "spill-to-disk" feature, not a "more RAM" feature. Use it when you can't avoid it, retrieve smartly when you can.

Reasoning models re-price your app

Standard models bill maybe 4-5x more for output than input. Reasoning models burn output tokens before the visible answer — thousands of "thinking" tokens that count as billable output. The published "$/Mtok output" looks comparable; the realized cost per request is much higher because you're generating 5-20x more output tokens than a standard model would.

Practical implication: if you're switching a workflow from a standard model to a reasoning model for a quality bump, your inference bill will not "go up a bit." It will probably 5-10x on the affected traffic. Worth knowing before the finance team finds out. The win comes when reasoning lets you collapse a multi-step pipeline (call → verify → revise → call again) into a single reasoning call — then you're spending more per call but fewer calls per outcome, and the math can swing back.

The Pareto frontier moves every quarter

The cost/quality frontier — the set of models where you can't get more quality without more cost or vice versa — shifts roughly every quarter. A model that was Pareto-optimal in Q1 is dominated by Q3. Sonnet-class capability at Haiku-class pricing in late 2024 became Haiku-class capability at sub-Haiku pricing by late 2025. The frontier is sliding down and to the right, continuously.

For your roadmap this means two things. One: if you priced your product assuming current model costs are permanent, you've overpriced. Two: a product feature that's "just barely affordable" today will be obviously affordable in six months. Build for the trajectory, not the snapshot.

Reading the Model Zoo Like a Product Person

You don't need an org chart of every frontier lab. You need a mental map of which lab is good at what, at what release cadence, and what that means for your product. Capability is converging; release cadence, niche, and pricing strategy are diverging. The PM-useful framing is to read the labs as portfolios, not as competing brand names.

Labs as portfolios

Anthropic — Claude family. The agentic-coding and safety leader. Strongest "drive a long-running tool-using agent without going off the rails" behavior, which is why Claude Code, Cursor's agent mode, and most of the coding-agent ecosystem default here. Release cadence is steady — roughly one major Sonnet bump every 2-3 months in the 4.x line, with 4.6/4.7 currently shipping. Pricing is middle-of-pack; positioning is "the model you trust with elevated tool access."

OpenAI — GPT and o-series. Product velocity leader and the lab with the most consumer distribution by an order of magnitude. GPT-5 sits at the frontier on most general benchmarks; the o-series continues to lead pure reasoning. They ship product surface (Operator, the agent SDK, voice, Sora) faster than anyone else, which is both their strength and the source of most of the "did they regress quality?" Reddit threads.

Google DeepMind — Gemini 2.5 family. The leaders on context length, multimodal (especially video and audio), and aggressive pricing. Flash is the de facto default for cost-sensitive workflows in 2026 — close to free at high volume. Ultra is genuinely competitive at the frontier. The product story is "tightest integration with the rest of Google's stack," which is enormous if your users live in Workspace.

Meta / DeepSeek — the open-weights frontier pressure. Llama keeps the floor honest; DeepSeek's V3/R1 line proved that open-weights reasoning can match closed labs, which dropped prices industry-wide. Their job in your mental model is "substitution threat" — they prevent the closed labs from charging whatever they want.

Mistral / xAI / Cohere / Qwen — wildcards. Mistral has a credible European-jurisdiction story. xAI ships occasionally-spiky models with deep X integration. Cohere is enterprise-RAG-focused. Qwen (Alibaba) ships strong open weights at high frequency. Treat these as situational picks rather than defaults.

"Frontier" is a Pareto curve, not a leaderboard

"Frontier model" doesn't mean "smartest." It means "no other model dominates it on both cost and quality." The frontier is a curve, and every lab ships somewhere along it. GPT-5 is on the frontier. Claude Opus 4.7 is on the frontier. Gemini Flash 2.5 is on the frontier in its weight class. They are not competing for one crown; they're staking out positions on the Pareto curve.

This reframes "which lab is winning" into "which point on the curve fits my workload." A consumer chat product cares about a different point than an agentic coding workflow than a high-volume classification pipeline. The right question is rarely "is GPT-5 or Claude better"; it's "what cost-per-quality point does my workload need, and which lab is currently closest to that point?"

Reasoning vs standard is a cost axis

Most labs now ship a "reasoning" variant alongside the standard one — Claude with extended thinking, GPT-5 with thinking budgets, Gemini Thinking, R1-style flips. Treat the reasoning toggle as a cost dial, not a quality switch. You're trading ~5-10x more output tokens for ~10-30% accuracy on hard tasks. Whether that's a win depends entirely on whether your task is bottlenecked on accuracy or on cost.

Practical rule: reasoning on for offline / asynchronous / verifiable workloads (code generation, document analysis, agent planning steps). Reasoning off for chat, summarization, classification, retrieval-grounded answers. If you're not sure, A/B it on the eval set.

Reading the price sheet

$/Mtok numbers are easy to misread. The translation that actually matters is "what does one user-session cost?" Math: take an average session, count input tokens (system prompt + history + retrieved context) and output tokens (response plus any reasoning budget). Multiply through. Now you have a number you can compare to your unit economics.

Two surprises every PM hits the first time they do this. One: reasoning models can quietly 10x your per-session cost via output volume. Two: prompt caching can quietly 5x your cost when it doesn't hit, because you're paying the warm-cache premium on every cold start. Build the spreadsheet, run it against your actual session distribution, then make pricing decisions.

Routing is how production AI products are actually built

Most serious AI products in 2026 use three to five models in a single user request. A typical Cursor-style request might do: a Haiku-class model classifying the intent, an embedding model finding relevant context, a Sonnet-class model drafting the response, an Opus or GPT-5 reasoning model handling the actually-hard branch, a small judge model verifying output. This isn't an exotic pattern — it's the default for anyone serious about cost-per-quality.

The product implication: hard-coupling your application code to one model is malpractice in 2026. The right abstraction is a router that picks the model per request based on intent, complexity, and cost budget. When the frontier moves — and it will, every quarter — you swap the routing config, not the application. This is the difference between a six-month migration and an afternoon.

Open weights as substitution threat

You almost certainly don't need to self-host. The ops cost is real, the latency tradeoffs are bad, and the closed-lab APIs are cheaper than your engineering time. But the existence of credible open-weights options at every weight class — Llama 4, Qwen, DeepSeek — is the load-bearing piece of your supplier strategy. It's the BATNA in every negotiation, implicit or explicit. The closed labs price knowing you could leave; that's why prices keep dropping.

The Product Surface

The primitives — prompts, tools, retrieval, agents, evals — are well-known. The interesting question is how they fit together, where each is the wrong primitive, and where the actual product judgment lives. Most AI products fail not because their team picked the wrong model but because they picked the wrong primitive for the job and then spent six months optimizing the wrong thing.

Prompt engineering is software engineering

The "prompt engineering" memes have aged poorly. In practice, prompts in a serious product are versioned, tested, regression-checked, and reviewed in PRs. They have authors and changelogs. They get diffed. They have unit-test equivalents (small eval sets that gate deploys). They get rolled back when they break.

Teams that treat the prompt like a config string in a YAML file end up with the same operational pathologies as teams that treated SQL queries that way in 2010 — silent regressions, undiscoverable knowledge, nobody knows why the working version works. The good teams have a prompt registry, a small framework for templating, a way to A/B prompt variants in production, and an eval suite that runs on every change.

When RAG is the wrong primitive

RAG is the default when you need the model to answer over a corpus it didn't see in training. It's the right tool when the task shape is "find facts and quote them" — support docs, legal clauses, codebase grep-and-quote. Embeddings retrieve, the model paraphrases or cites.

RAG is the wrong primitive when the task is "reason over the whole corpus." Try asking RAG "what are the three biggest themes in our customer feedback this quarter" and watch it confidently summarize whichever five chunks happened to score high on cosine similarity. The chunks that didn't make the top-k don't exist for the model. You've replaced "the model knows everything" with "the model only sees what your retrieval scored highly," and your retrieval system is almost certainly worse at picking relevant context than the model would be at attending to a full set.

For reasoning over a corpus, options in order of preference: (1) put the whole thing in context if it fits, (2) use a hierarchical summarization pass that the model controls (let it pick what to drill into), (3) use an agent that does retrieval as a tool with feedback, (4) only then fall back to plain RAG. The plain "embed and stuff top-5 chunks" pattern is the right answer to far fewer questions than people use it for.

Agentic loops: the harness is the product

When you build an agent, the model is doing maybe 30% of the work. The other 70% is the harness — what tools you expose, how you describe them, what scratch space the model has, how memory persists across steps, how failure is detected, how the loop recovers, whether subtasks run in parallel, what the human-in-the-loop affordances are.

Look at the difference between Claude Code and a homegrown "I gave the LLM a bash tool and a goal" agent. Both have access to roughly the same capability. The Code version ships because the harness is good: tool schemas are tuned to the model's idioms, file operations are atomic and undoable, the planning loop has explicit checkpoint behavior, the memory is structured. The homegrown version is a research demo because the harness is naive.

For PMs: when you scope an "AI agent" feature, the model choice is one decision and the harness is twenty. Budget accordingly. The team that wins at agents is the team that's good at the unsexy infrastructure — tool taxonomy, retry semantics, failure observability, sandbox design.

Parallelism in agent design

Modern agent harnesses run subagents in parallel. The orchestrator agent spawns three search agents that each take different angles, then synthesizes. You get latency wins (parallel I/O), quality wins (more attempts, ensemble synthesis), and cost surprises (3-5x more model calls per "request"). The cost math swings in your favor when the parallelism replaces a longer serial chain — three parallel attempts at ~10s each beats one serial chain of 45s on both quality and clock time, even if you spent 2x the tokens.

The eval crisis

Most public benchmarks are saturated or close to it. MMLU is decorative. HumanEval gets a 95+ from anything that can spell. Even Arena-Hard and the new generation of "hard" suites top out faster than they're built. Benchmarks correlate with product success at roughly the level your team's gut does — which is to say, weakly.

What works for product evals: hand-curated golden sets, 50-300 examples deep, representing the actual distribution of your hard cases. Real user inputs (anonymized, sampled). Outputs scored against expected behaviors, not exact strings. Track score per cohort (new-user, power-user, edge-case-X). Re-baseline whenever the model changes. Treat the eval set the way the security team treats the audit log — a living artifact, owned, never stale.

LLM-as-judge is the trick of using a strong model to grade your production model's outputs. It works well for subjective dimensions ("is this answer helpful," "did it follow the cited source"). It breaks when the judge has the same biases as the model under test, when the criteria are subtle, or when judges drift between runs. Use judges for cheap continuous monitoring; never let them be the only signal on a critical metric.

Read Hamel Husain if you haven't. The thesis: evals are not the boring infrastructure you build at the end. They are the spine of every other decision — model selection, prompt iteration, deploy gates, vendor switches. If you don't have them, you're flying on vibes; if you have weak ones, you're flying on biased vibes.

The Hard Parts No One Wants to Admit

The first five chapters are the stuff your ML lead would tell you over coffee. This one is the stuff they'd tell you over the second drink — the failure modes that don't show up in pitch decks, and the strategic questions every AI product leader has to answer honestly. None of these have clean solutions. The teams that succeed are the ones that recognize them early and decide deliberately, instead of getting eaten by them later.

Eval debt is the silent killer

You can ship faster than you can evaluate. That sentence is the single most important thing in this whole document, so I'll repeat it from a different angle: the first version of your feature shipped in two weeks because the prompts wrote themselves. The third version shipped in a day because you reused components. But you have no idea if v3 is better than v2.

Eval debt compounds because every shipped change makes the surface area harder to evaluate. By month six the team is arguing about whether the model "got worse" and there is no way to resolve it. The cure is unfun: pause new feature work, build the eval harness, gate every change. Teams that pay this tax up front accelerate forever after. Teams that don't end up in vibes-driven development, which is exactly as load-bearing as it sounds.

Model migrations break your prompts

You shipped on Claude Sonnet 4.5. Six months later, 4.7 lands — cheaper, smarter, faster. You upgrade. Three of your prompts subtly break. The eval set you don't have can't tell you which. Your customers notice before you do.

Prompts that worked perfectly on the old model can fail on the new one in non-obvious ways: different default formatting, different sensitivity to instructions, different refusal patterns, different reasoning depth at the same token budget. Every migration is a small product release. Hard-coupling your application to a specific model version is the AI equivalent of writing all your code against one database vendor's stored procedures — possible, common, and a strategic time bomb.

The right architecture: an abstraction layer where the model is configurable per call, prompts are versioned per model where they need to be, and the eval suite runs against every candidate before any migration ships. This sounds heavy. It costs roughly two weeks of engineering up front and saves six months over the product's lifetime.

Hallucinations are a UX problem

Hallucinations are not going to be solved by a smarter model. The frontier has been chasing this for years; the rate gets better, never zero. The right move is to stop treating it as a technical defect and start treating it as a product surface to design around.

Good moves: cite sources whenever possible, design UI that makes uncertainty visible (confidence bands, "this looks unverified" badges), force the model to ground answers in retrieved content with verifiable citations, give the user obvious "this is wrong" affordances that feed back into your eval set, fall back to "I don't know" gracefully when grounding fails. The product question isn't how do we eliminate hallucinations. It's when the model is wrong, how do we make sure the user catches it before acting on it?

If your product is in a domain where wrong answers have consequences — medical, legal, financial — your hallucination strategy is your product strategy. Build it first.

When fine-tuning is right

Almost never for product features. Often for cost / latency squeezes after you've nailed prompts. The honest framing in 2026: fine-tuning was the answer in 2022 when models were dumb. Today's instruct models are smart enough that a thoughtful system prompt and good RAG get you 90% of what fine-tuning would, with none of the migration pain and the ability to swap models freely.

Fine-tuning earns its keep when: you have a narrow task with stable schema (extract this exact structure from this exact document type), thousands of high-quality labeled examples, and a latency or cost target that prompting the big model can't meet. In that specific corner of the world, you take a strong teacher's behavior and distill it into a smaller model you control. Outside that corner, fine-tuning is mostly an expensive way to make your migration story worse.

The triangle in real decisions

Cost, latency, quality. Pick two. The reason this meme persists is that every nontrivial product choice in AI is a trade between two of the three. Added a verifier step? Traded cost and latency for quality. Switched to a smaller model? Traded quality for cost. Enabled streaming? Traded total latency for TTFT. Enabled reasoning? Traded cost and latency for accuracy on hard cases.

The product question is never "which is best." It's "which two are we optimizing for on this surface, and is that the right two?" A consumer chat product optimizes latency and cost. A coding agent optimizes quality. A real-time voice agent optimizes latency above almost everything. Write the answer down where the team can see it; revisit it quarterly.

Capabilities don't exist yet — plan for shipping into a moving target

Anthropic and OpenAI ship a meaningful model bump roughly every three months. The capability surface you're building on today will be obsolete-or-cheap by Q3. This means the product question is unusually short-horizon: what's the half-life of my moat?

If your moat is "we have the only product doing X with current model capability," the half-life is one quarter. If your moat is "we have the only product doing X with our proprietary data and a workflow that compounds with model improvements," the half-life is years. The good news: the second class is much larger than people think, because most AI feature ideas are downstream of "the model can do Y now," and the long-tail of useful applications of Y is enormous. The bad news: the first class is what most "AI features" actually are.

AI feature vs AI-first product

An AI feature is a chat sidebar bolted onto an existing product. A summarize button. An autocomplete. A "draft with AI." These are becoming table stakes — useful, expected, increasingly commodity. The competitive advantage from shipping them is real but short-lived, because everyone is shipping them.

An AI-first product has a core loop that only makes sense because LLMs exist. Cursor is AI-first; without a model it's a text editor. Granola is AI-first; without transcription it's an empty notebook. Perplexity is AI-first. The whole agentic-coding ecosystem is AI-first. The differentiation isn't the AI; it's the workflow the AI made possible.

The alpha is in the second category, and the gap between the two is widening. AI features are getting commoditized into the platforms (Microsoft, Google, Apple are absorbing the obvious ones). AI-first products are inventing new workflows the platforms can't easily replicate. If you're starting something now, ask: is the core loop here possible without an LLM? If yes, you're shipping a feature, and you're racing the incumbents. If no, you have a chance at something durable.

The meta-skill: iteration speed across the stack

The most valuable AI product skill in 2026 isn't model picking, prompt engineering, or even eval design — it's iteration speed across the whole stack. A team that can change a prompt, run evals, deploy, observe production, learn, and ship the next iteration in hours wins against teams that take weeks. Models will keep changing. Costs will keep dropping. Capabilities will keep expanding. The teams that built fast-iteration infrastructure in 2024 are eating the lunch of teams that picked the "perfect" model in 2024 and shipped it once.

If you take one operating principle from this primer: build the iteration loop first, then build the product on top of it. Your evals are the loop. Your prompt versioning is the loop. Your observability is the loop. Your model-swap flag is the loop. Everything that lets you change one thing and measure the impact is the moat. The product is everything you ship through it.