How do you know when to build an LLM?

Build the least model your problem forces you to. Start with a hosted model behind good prompting and retrieval, fine-tune an open model only when specific behaviours stay wrong after that work, and train from scratch only when no open model can serve your language, domain, or data-residency constraint. Most teams never reach the third step.

How do you build an LLM from scratch?

You assemble a corpus of trillions of tokens, run a large cluster of accelerators for weeks, and train the model from random weights using published recipes. The scaling work behind Chinchilla suggests roughly 20 tokens of text per model parameter for compute-optimal training. The hardest and least documented part is the data pipeline, which almost never appears in the tutorial.

How do you build your own LLM without training from scratch?

You fine-tune an existing open model on your own examples. Parameter-efficient methods such as LoRA freeze the original weights and train a small set of new ones, which the LoRA paper reports can cut trainable parameters by up to 10,000 times against full fine-tuning of a 175-billion-parameter model. That makes a custom model achievable on rented hardware instead of a dedicated cluster.

What does it cost to build an LLM?

The cost shows up as compute and data, and then as time. Compute is the visible number you can price in advance, but data cleaning usually consumes the most human effort, and time is the currency that sinks projects, since hosted baselines keep improving while a from-scratch run is still training. Fine-tuning a small open model can cost a few hundred dollars; a frontier model from scratch runs into the tens of millions.

Do most companies need to build their own LLM?

No. Most products need an application built on top of a hosted model. Retrieval and prompt design solve the majority of what teams describe as needing a custom model, and they cost a fraction of training one. Reserve model-building for cases where a measured failure of the hosted options forces it.

What is the difference between building an LLM and building an LLM app?

Building an LLM means creating or adapting the model weights themselves. Building an LLM app means creating the product around a model: the retrieval and prompts that make it reliable. The two share three letters and almost nothing else, and nearly all the value most teams create lives in the app layer.

When to Build an LLM, and When to Fine-Tune Instead

When to build an LLM has a shorter answer than most teams want to hear. You almost never build one from scratch, and more often than you would guess, you should not build one at all. The useful version of the question is which path fits your problem and how far down the stack you actually need to go.

A large language model is a system that predicts text one token at a time after learning statistical patterns from a huge body of writing. "Building" one can mean three very different amounts of work, and picking the wrong one is how six-figure budgets quietly evaporate.

Here are the three paths, ordered by how much of the model you own:

Training from scratch, where you choose the architecture and run the full training job on data you assembled yourself.
Adapting an open model, where you take someone else's trained weights and fine-tune them on your own examples.
Wrapping a hosted model, where you call a model through an API and build your product around it.

Most teams who say they want to build an LLM want the third option, and occasionally the second. The first is a research programme wearing a product roadmap.

What building an LLM actually means

Owning more of the model buys you control and costs you everything else. The trade is close to linear, so the honest first question is how much control the problem genuinely requires.

Training from scratch means starting with random weights and teaching the model language itself, from what a sentence is up to how your particular domain talks. You need a corpus measured in trillions of tokens, a cluster of accelerators, and a team that can diagnose a training run which silently diverged at 3 a.m. on a Sunday.

Adapting a model starts from weights that already speak fluent English and nudges them toward your task. This is fine-tuning, and it is where the phrase "build your own LLM" is usually both honest and affordable.

Wrapping is the shallowest depth and, for most products, the correct one. The model stays on someone else's servers, and your engineering goes into the prompts and retrieval that make the whole thing reliable. If you have never trained a model, this is where to start, and a plain-language walk through the moving parts is what a beginner's guide to working with Claude is for.

Building LLM apps versus building the model

The most common confusion hides inside the word "build." Building LLM apps and building an LLM are different disciplines that share three letters and almost nothing else.

An LLM app is the product layer. Retrieval that feeds the model the right context, and prompts that hold up under strange input. This is real engineering, and it is where nearly all the value most teams create actually lives.

Building the model is the layer underneath, meaning the weights themselves. Very few products need to touch it, because the hosted models are already strong at general language and improving faster than any in-house effort could match.

A useful test: if you can describe your problem as "the model needs to know our stuff and follow our rules," you have an app problem. Retrieval and prompt design solve it, and the craft of that prompt layer is its own deep subject, covered well in building prompt systems that hold up over time.

How to build an LLM from scratch

Say you have decided you genuinely need your own weights. Here is what building an LLM from scratch demands, in roughly the order the costs hit you.

First, data. The scaling work in DeepMind's Chinchilla paper found that compute-optimal training wants around 20 tokens of text for every parameter in the model. Their 70-billion-parameter model, trained on four times more data than the 280-billion-parameter Gopher it beat, made the case that most models had been undertrained, given too little data for their size.

Scale that up and the figures turn serious. Meta's Llama 3 herd put its flagship at 405 billion parameters trained on more than 15 trillion tokens. That is a training bill in the tens of millions of dollars, incurred before anyone has evaluated a single output.

Second, compute: thousands of accelerators running for weeks, plus the interconnect and the engineers who keep the run from falling over. Third, the training itself, which is the part with the most published guidance and the smallest share of your total time.

The counterintuitive part is that writing the training code is the easy stretch. The open-source recipes are mature. What separates a working model from an expensive random-number generator is the data pipeline, and that work is unglamorous and almost never in the tutorial.

The deep end of this work, from tokenisation through pretraining objectives to running the job on real hardware, is the subject of training and building large language models. It is the one path here that genuinely rewards a book-length treatment.

A developer working at night on dual monitors, writing code to train a language model — Photo by cottonbro studio on Pexels

How to build your own LLM by adapting one

For most teams that want a model of their own, the real answer is fine-tuning, which starts from an open model and adjusts it on your examples. You get a model that behaves like yours without paying to teach it English from scratch.

The economics changed with parameter-efficient methods. The LoRA paper showed you can freeze the original weights and train a small set of new ones, cutting trainable parameters by up to 10,000 times against full fine-tuning of a 175-billion-parameter model, and reducing the GPU memory requirement to about a third. A job that once needed a server room now fits on hardware you can rent by the hour.

Fine-tuning earns its place when you need a consistent output format, a specific tone, or reliable behaviour on a narrow task that prompting alone keeps getting wrong. It is weaker at teaching genuinely new facts, which is a job retrieval handles better and more cheaply.

The trap is reaching for fine-tuning first. Try a strong hosted model with good prompting and retrieval before you fine-tune anything, because a large share of "we need a custom model" turns out to be "we needed a better prompt." Building agents and tools on top of a capable base is its own craft, treated in depth in designing and building LLM-based agents.

The cost of building an LLM: data, compute, time

The bill for building an LLM comes in three currencies, and the one people underestimate is not the money.

Compute is the visible cost and the easiest to plan for, since you can price accelerator hours before you begin. Data is the hidden cost. Cleaning and filtering a corpus is where most of the human months go, and a model trained on mediocre data is mediocre in ways no amount of compute repairs.

Time is the currency that sinks projects. A from-scratch effort is measured in quarters before you even know whether the result is competitive, and hosted models do not pause to let you catch up. By the time a scratch-built general model is ready, the baseline it needed to beat has usually moved.

This is why the decision deserves real scrutiny. The costs are large and front-loaded, and they land in full before you have any evidence the approach will work.

A framework for deciding when to build

Here is the decision compressed to a rule of thumb. Start at the cheapest path and move deeper only when a specific, measured failure forces you.

Walk it in order:

Start with a hosted model behind strong prompting and retrieval. Ship it and measure the failures.
If particular behaviours stay wrong after real prompt and retrieval work, fine-tune an open model on those cases.
Consider training from scratch only when you have a defensible reason the open models cannot serve, such as a language or domain they do not cover, or a hard data-residency constraint.

The through-line is that building an LLM is a decision about the lowest layer you are forced to touch. Almost every team is forced less deep than they first assume, and the ones who skip the framework tend to discover this the expensive way.

So the close is a single question to carry into the meeting. Ask what the shallowest layer that solves the problem is, because that turns a budget question into a question about your users. Answer it honestly and the build-or-buy call usually resolves itself.

When to Build an LLM (and When to Skip It)