Software & AI · Stack decisions · Written and maintained by Haink’s AI adoption team · Updated August 2026 · 8 min read
Build vs Buy: Custom AI/LLM Application or Off-the-Shelf?
Most AI buying decisions are not really build-versus-buy — they are build-on-top-of decisions. You will almost never train a foundation model from scratch; the real question is how much of your competitive advantage lives in the AI layer, and how much is undifferentiated plumbing you should rent. Buy the generic parts, build the parts that create advantage from your own data and workflows.
Key takeaways
- Buy off-the-shelf for generic, non-differentiating use cases where a mature product already exists.
- Build custom when the AI touches your core differentiation, needs your proprietary data, or must run privately.
- The common answer is hybrid: a thin custom layer (retrieval, workflow, integration) on top of bought foundation models.
- You rarely need to train a model from scratch — building on foundation models is far cheaper.
- The layers you must build are the unglamorous ones — ingestion, retrieval logic, evaluation and integration. The layer teams most want to build, the model, is the one to rent.
- Per-seat pricing compounds: against a $100,000 build, break-even is around 271 seats in year one and about 132 over three years.
- Decision test: would this feature, working perfectly, be a reason customers choose you? If yes, build that part.
This page is about how, not whether. Whether a use case should be built at all — differentiation, data, portfolio sequencing — is the strategic call, and it is made in build vs buy AI: the strategic decision. Read that one to decide whether and where. This one assumes the use case is greenlit and asks which parts of the system you assemble yourself.
On this page
The stack, layer by layer
An LLM application is eight or nine layers, and build-versus-buy is decided per layer rather than once. Set out that way, the answer stops being a matter of philosophy: some layers are solved problems that would be absurd to rebuild, and some encode knowledge nobody can sell you.
| Layer | Default | Why | Cost of getting it backwards |
|---|---|---|---|
| Foundation model | Rent | Training one is a nine-figure exercise with no payback for a company whose product is not a model | The entire budget, spent to obtain a model worse than the one you could have called over an API |
| Inference serving | Rent, then own | Own it when the volume arithmetic says so, not before — the crossover is near 340 tokens/sec sustained | A node at 15% utilisation with no payback story |
| Vector store | Adopt | pgvector, Qdrant, Weaviate and Milvus are mature; writing one is a hobby, not a project | Months spent on a solved problem |
| Ingestion and chunking | Build | Specific to your documents, and the single largest lever on retrieval quality | A generic splitter you cannot tune, halving recall you will then blame on the model |
| Retrieval logic | Build | Hybrid weighting, metadata filters and permission filtering are specific to your corpus and your access model | An assistant that answers from documents the asker is not cleared to read |
| Evaluation set | Build, always | It encodes what “correct” means in your domain. No vendor can sell you that | Shipping on intuition, with regressions invisible until users report them |
| Guardrails | Rent generic, build specific | Prompt-injection filtering is commodity; what counts as out of scope is yours | Either a system that refuses everything, or one that answers anything |
| Workflow and integration | Build | The wiring into ERP, CRM and document systems — where most of the budget actually goes | Assuming a purchase removes this. It removes the model, not the wiring |
| User interface | Embed, rarely build | People want answers inside the tool they already have open | A new destination nobody visits |
Read the Build rows together and a pattern appears that most roadmaps get backwards. Every layer you must build is unglamorous — parsing awkward documents, tuning retrieval, writing an evaluation set, wiring into a system of record. The layer teams are most eager to build is the model, and it is the one to rent. Projects fail on that inversion more often than on any technical choice.
When per-seat pricing loses
“Owning it is cheaper at our volume” gets asserted far more often than it gets calculated. It has a simple form: a purchased tool costs per seat per year and scales linearly with adoption; a build costs a fixed sum plus a smaller annual run cost, and does not.
Take a production LLM application at $100,000 to build with about $30,000 a year to run, against a seat priced at $40 per user per month — $480 a year:
Two honest caveats, because a table like that is easy to misuse. It compares cost only — the purchased product exists this week and the built one exists in three months, and the build carries execution risk the purchase does not. And it assumes adoption you have not yet proven; buying fifty seats to find out whether anyone uses the thing is a cheaper experiment than building for five hundred and discovering they do not.
Which suggests the sequence rather than the answer: buy small to establish that the workflow is real, then build once seat count and usage are measured rather than forecast. The hidden costs on both sides — and there are several on each — are set out in the strategic build-vs-buy page; cost ranges by project type are in how much custom AI costs.
When to buy off-the-shelf
- The use case is generic — general transcription, standard OCR, common chat support — and a mature product does it well.
- Speed matters more than fit, and you can live with the vendor's roadmap, pricing and data policies.
- Volume is low enough that per-seat or per-call pricing stays affordable.
- The capability is not where you compete — it is table stakes.
When to build custom
- The AI touches your core differentiation — proprietary data, workflows or domain expertise competitors don't have.
- You need it grounded in your own data and embedded in your systems, not a generic tool sitting alongside them.
- Data residency, privacy or compliance require private or on-premises deployment.
- At your volume, owning the system is cheaper over time than per-seat SaaS.
- No off-the-shelf product fits the specific shape of your problem.
At a glance
| Factor | Lean buy | Lean build |
|---|---|---|
| Differentiation | Commodity capability | Core competitive advantage |
| Data | Generic / public | Proprietary, must be grounded in it |
| Privacy | Vendor cloud acceptable | Must stay in your network |
| Volume | Low or unpredictable | High and steady |
| Fit | A product matches your need | Your problem is uniquely shaped |
| Time-to-value | Need it this week | Worth weeks for the right fit |
The hybrid reality most companies land on
In practice the best answer is usually a thin, custom layer on top of bought foundations. You don't train a language model — you use proprietary or open-weight foundation models and build the retrieval, workflow, evaluation and integration that make them useful for your specific problem. That custom layer is where engineering pays off and where off-the-shelf products can't follow you, while the expensive, commoditized base model is rented.
Common mistakes, by root cause
Four of these account for most failed LLM projects, and each has the same visible symptom — the project ran long and delivered less than promised — which is why they are worth separating by cause.
| What happened | Root cause | What to do instead |
|---|---|---|
| Built the commodity. Months rebuilding transcription, OCR or a vector store that a mature product already does well | Layer-by-layer decisions were never made; the whole system was classed as “build” in one go | Decide per layer using the table above. Rent anything a competitor could buy tomorrow |
| Bought the differentiator. The one capability that should set you apart is now a vendor's product every competitor can also license | The differentiation question was answered at the project level rather than the layer level | Rent the model; build the retrieval, the evaluation and the wiring into your workflow |
| Trained from scratch. A custom model where retrieval and prompting on a foundation model would have done | Confusing “the model does not know our data” with “the model must be retrained” | Work the ladder in order — prompting, then retrieval, then fine-tuning. See RAG vs fine-tuning |
| Ignored run cost. Per-seat SaaS chosen without modelling it at full adoption, or hardware bought before volume was measured | The build-versus-buy case was made once, at pilot scale, and never recomputed | Compute the seat break-even above, and the inference break-even in on-premises vs cloud |
A fifth is quieter and more common than any of them: treating “buy” as instant. A purchased tool that is not integrated into the workflow and fed the right data returns close to nothing. Buying removes the model-building; it does not remove the ingestion, the permissions, the integration or the change management, and those are where most of the calendar goes either way.
Building AI software on your own infrastructure?
Model, pipeline and GPUs under one contract — tell us the use case and we'll scope it.
Talk to our engineers Prefer email? sales@haink.org
What to read next
- Build vs buy: the strategic decision — if you have not yet settled whether to build this use case at all
- How much does custom AI cost? — ranges by project type, once you know which layers are yours
- RAG vs fine-tuning — which technique the custom layer should use, with the volume at which each pays back
- How to build a production RAG system — the build layers in detail, once the decision is made
Related Resources
- LLM Applications & RAG
- Software & AI Development Services
- AI Solution Blueprint — a portable specification you own, implementable by any competent team
Frequently Asked Questions
Should we build or buy an AI solution?
Buy off-the-shelf for generic, non-differentiating use cases where a mature product exists. Build custom when the AI touches your core differentiation, needs your proprietary data and integrations, or must run privately. Most companies do both: a custom layer on top of bought foundation models.
Do we need to train our own model?
Almost never. Most production systems use proprietary or open-weight foundation models with retrieval (RAG) and, where justified, light fine-tuning — not models trained from scratch.
Is building a custom LLM app expensive?
Building on top of existing foundation models is far cheaper than training from scratch — you invest in retrieval, workflow, evaluation and integration, not in training a base model. Cost scales with complexity and integration depth.
How do we decide which parts to build?
Build the parts that create competitive advantage from your data and workflows; buy or rent the undifferentiated components around them. The test: would this feature, working perfectly, win you customers?
Which parts of an LLM application should we never build?
The foundation model, the vector store and the user interface, in almost every case. Training a foundation model is a nine-figure exercise with no payback for a company whose product is not a model; pgvector, Qdrant, Weaviate and Milvus are mature enough that writing your own is a hobby rather than a project; and users generally want answers inside a tool they already have open rather than in a new destination. Rent anything a competent competitor could buy tomorrow.
Which parts should we always build?
Ingestion and chunking, retrieval logic, the evaluation set, and the integration into your systems of record. Each of these encodes something specific to you — how your documents are shaped, who is allowed to see what, what a correct answer means in your business, and how the result reaches the person who acts on it. None of them is a product anyone sells, and all four are unglamorous, which is why roadmaps consistently under-resource them while over-resourcing the model.
At what point does building beat per-seat pricing?
For a production application costing about $100,000 to build and $30,000 a year to run, against a seat at $40 per user per month, break-even is roughly 271 seats over one year, 132 over three and 104 over five. But that compares cost only: the purchased product exists this week and the built one exists in three months, and the calculation assumes adoption you may not have proven. Buying a small number of seats to establish that the workflow is real is usually a cheaper experiment than building for a headcount you have forecast rather than measured.
What is the biggest build-vs-buy mistake?
Building the commodity and buying the differentiator — rebuilding generic capabilities while outsourcing the one thing that should set you apart.
Sources and scope. The seat break-even is arithmetic from stated assumptions — a $100,000 build with $30,000 annual run cost against a $40 per user per month seat — and every one of those should be replaced with your own figures; the formula is the durable part. Layer defaults are engineering judgement rather than measurement, and the right answer changes where a layer genuinely encodes something proprietary. The comparison is cost-only and does not price execution risk, time to value, or the possibility that adoption differs from forecast.
Reviewed: August 2026.