Custom AI Agents: What to Build and What to Buy

Build the parts that touch your data and your money; buy everything else. In practice that means renting the model, buying or reusing the runtime that runs the loop, and building the tools, the permission boundary and the approval-and-audit layer yourself. A build-or-buy decision can be right in general and still wrong for the layer it was applied to.

I build AI agents that work inside business systems, so read this with that bias in mind. It is just as useful if you buy a platform, hire someone else, or decide an agent is not worth it yet.

"Custom AI agents" is five decisions wearing one name

An agent is not a product you can compare on a pricing page. It is a stack, and each layer has its own build-or-buy answer, cost curve and failure mode. When a vendor demo blurs the layers together, "should we build this?" becomes unanswerable, because the honest answer is "which part?"

Layer	What it decides	My default	Build only when
Model	Reasoning quality, cost per action, where your data goes	Rent it	You are in the model business
Runtime and orchestration	The loop: plan, call a tool, read the result, retry, stop	Buy or reuse	You need approval semantics or network access the platform cannot give
Tools and connectors	The verbs the agent can perform on your systems	Build	Almost always
Data, memory and permissions	What it may read and remember, for which tenant and role	Build inside your perimeter	Always
Controls	Approval gates, audit trail, evaluations, kill switch	Build	Always, even on a bought platform

The mistake to avoid is picking one answer and applying it to all five. That looks like buying an "agent builder" when the missing piece was a connector, or building an orchestration loop when the platform already had one and the tools were the actual work.

The model: rent it, and stop treating it as the decision

You will not train a foundation model, and for a business agent you should not try. Model choice is a cost and data-residency decision, not a strategy one, and it is reversible quickly if the rest of the stack is provider-neutral.

Pricing is metered per token. Anthropic's published API pricing lists Claude Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens, with Haiku 5.5 at $0.10 per million input tokens for prompts up to 100,000 tokens (Anthropic pricing, checked October 2026). That gap is your most useful lever: use the cheap model to classify, extract and route, and the stronger one to plan and write.

Three things move the bill on any model: retries and loops, because a task that fails twice is paid for three times; context you resend every turn; and output length, priced several times higher than input on every tier.

Self-hosting an open model is a different cost structure: no per-token meter, but you own the hardware, upgrades, evaluation harness and on-call. It is justified when data cannot leave your environment or volume is enormous, rarely as a way to save money at pilot scale.

The trap is fine-tuning before you have an evaluation set. Without fixed cases and expected outcomes you cannot tell whether the model got better or merely different, and you will keep the change because it cost money.

The runtime: buy it, unless the loop is your product

The runtime is everything between "the model decided to act" and "the action happened": the tool-calling loop, retries, state, tracing, streaming, and somewhere to pause for a human. Writing your own is real engineering time that produces nothing a customer can see.

Hosted agent platforms also make the economics visible in a way a demo hides. Microsoft's Copilot Studio billing rates meter agent work in Copilot Credits by feature: a classic answer costs 1 credit, a generative answer 2, an agent action 5, grounding in your tenant graph 10, and agent flow actions 13 per hundred (Microsoft Learn, billing rates, checked October 2026). Read that table as a business model: outside the employee use that Microsoft 365 Copilot licences include at no charge, the platform charges per unit of work, including per answer.

Two consequences follow:

1. Volume decides, not model quality. At a few hundred actions a month, per-action pricing is trivial against engineering time. At tens of thousands, the same table is the largest line in the project.
2. The cheapest architecture needs fewer actions. One tool that does "check stock and reserve" in a call beats four conversational round trips, on any platform.

I would still buy the runtime by default. Build your own only when the actions must execute inside your network against your database with the asking user's permissions; you need approval per action with a named approver and the platform offers only all-or-nothing autonomy; you must swap model providers without re-platforming; or per-action cost at your measured volume exceeds what the engineering would cost to own.

The tools: this is where custom work earns its keep

A tool is a named verb with typed input, a fixed risk level and a result the model cannot misread: invoice.create_draft, inventory.check_availability, payment.propose_refund, ticket.add_note. The agent's capability is exactly the set of verbs you give it, and nothing more.

This is the layer to build, and it is usually most of the engineering. Vendors cannot ship it for you, because their connectors are generic by design: a "create record" action that works for every customer cannot know your credit limits, approval thresholds, tax treatment or which fields your accountant will reject.

Rules I apply to every tool:

• One verb per tool. "Manage the order" is four tools pretending to be one, and the model will pick wrong.
• Risk is fixed in code. A tool that posts a journal entry is critical every time; the model does not get to decide it is routine today.
• Typed input, validated before execution. If the model can pass free text where an ID belongs, it eventually will.
• Return state, not prose. Give back the record id, status and changed fields, so the next step is grounded.
• Idempotency keys on anything that writes. A retried call must not create a second invoice.
• No SQL, no shell, no generic API passthrough. Each turns one bad instruction into a data incident.

A good tool layer is also what you keep if you later change platform, model or vendor. That is the strongest argument for building it: it encodes your business rules and outlives every other choice in the stack.

Data, memory and permissions: build the fence before the feature

Retrieval is where agents quietly become a data-protection problem, and the failure is almost always scope. An agent wired to "the organization's knowledge" reads whatever is in that corpus, which in a real business includes contracts, HR notes and other customers' records. If the agent is reachable from a public surface, it eventually answers the wrong audience with the right words.

Build the boundary before the feature:

• Scope retrieval by an allowlist of sources, never by "everything this tenant owns". Public answers come from public sources only.
• Scope by the asking user's role, not by the service account the agent runs under.
• Decide what memory means. If it remembers across sessions, that memory is a store you control, with a retention period, an owner and a way to delete a person's records.
• Treat everything it reads as data, never instructions. An email, web page or uploaded PDF can contain text written to manipulate the agent. An agent reading untrusted content should not also hold write access to money, or credentials broader than the task.

None of this is model work. A platform can enforce these boundaries, but only you can define them, because they depend on your systems.

Controls: the layer people buy a platform to avoid, then build anyway

Every agent touching a business system needs the same six controls, bought or built:

1. A policy gate. Each proposed action is checked against an allowlist, a risk ceiling and a rate limit before anything happens.
2. A hard line for critical actions. Posting a journal, registering a payment, issuing a refund and approving a purchase order never execute automatically, at any autonomy setting.
3. An append-only ledger. What the agent read, proposed and executed, and who approved it. You need this for the first dispute and every audit after it.
4. An evaluation set. Fixed cases with expected outcomes, run before every model, prompt or tool change. Without it, "it feels better" is the whole quality process.
5. A kill switch that works in one step, per agent, not per deployment.
6. Fail-closed sending. If the suppression or opt-out check errors, nothing goes out. An error in the gate means "do not send", never "send anyway".

When you evaluate a vendor, ask for the log of everything the agent did yesterday, per-action approval rather than a global autonomy switch, and a one-click stop. A platform that cannot show you those three has not solved the hard part; it has moved it to you with less visibility.

The decision in five questions

Run these in order.

1. Does the agent write to a system of record? If it only reads, summarises and drafts, buy. The mistake cost is contained and a bought tool is enough.
2. Do you already pay for a platform that can run this? Then trying it there costs almost nothing, and it is the right first experiment.
3. Are the actions generic or domain-specific? Calendar, email, documents and tickets are generic: buy. Invoices, stock, credit limits and production orders are yours: build the tools, rent the rest.
4. What volume will it actually run at? Model actions per month against per-action pricing. If you cannot estimate volume, you are ready to pilot, not to choose.
5. Can you staff the controls? If nobody owns the exception queue, the ledger and the evaluations, buy a platform that ships as much of them as possible and narrow what the agent may do until someone can.

What each side actually charges for

	Buying a platform	Building
Upfront	Subscription, setup, integration to your systems	Design, tool engineering, controls, evaluations
Per use	Per action, credit or seat	Tokens, infrastructure, and your on-call
When you need a change	The vendor's roadmap and release cycle	Your backlog, your priority
When it fails	Vendor incident, your customers, your reputation	Your incident, your on-call
When you leave	Export formats, who owns the configuration	You own it, including the maintenance

The row people underweight is the last one. Ask before signing what you can take with you: prompts, tool definitions, evaluation cases, logs. If the configuration lives in the platform, you are renting the agent's brain as well as its runtime.

What goes wrong, and how to spot it early

• The pilot works and production stalls. Nobody owns the exceptions. An agent that is right most of the time creates a queue, and someone has to work it.
• Accuracy looks high and the output is still unusable. With no per-action risk ceiling, a mostly-accurate agent is trusted with the small share of cases that matter. On money, "mostly right" is wrong.
• The tool surface grows after launch, usually because someone granted broader permissions to stop a bug. Every added verb is a new failure path.
• Nobody can answer what it did yesterday. No ledger, or one that records the final state without the proposal and the approver.
• The evaluation set is three manual tries. Every model upgrade becomes a coin flip, and the team learns to avoid upgrading.
• It has write access because read-only felt slow. The most expensive shortcut in the stack, and the easiest to take under deadline pressure.

The order I would do it in

1. Write the stack down for your case, five layers, one sentence each. A row you cannot fill is your risk.
2. Pick one task where a mistake is cheap and visible — a draft, a summary, a classification, a read-only lookup.
3. Build the tools and the ledger first, even if the runtime is temporary for a week. They are the parts you keep.
4. Rent the model and reuse the runtime. Route with the small model, decide with the strong one, keep the provider behind an interface.
5. Write the evaluation cases before the second feature. Ten real cases with expected outcomes beat a demo.
6. Revisit buy versus build once you know the volume. The per-action price you can then calculate beats any estimate made before launch.

If you want help working out which layers of your own stack to build, and putting approval and audit controls around the ones that touch money, that is what my AI integration engagement covers: an opportunity assessment, the solution architecture, and a working prototype connected to your systems.

For the narrower question of which sales tasks an agent should be allowed to touch at all, AI sales agents: where they help and hurt sets out the same gating logic task by task.

Frequently Asked Questions

Should I build my own AI agent or buy a platform?

Buy the model and the runtime; build the tools, the data boundary and the controls. If the agent only reads, summarises and drafts, a bought platform is usually enough. If it writes to your ERP, the tool layer encodes your business rules and is worth owning.

How much does a custom AI agent cost to build?

It depends on the number of tools, not the number of conversations. The cost drivers are how many systems the agent must write to, how much of the permission model you have to build, and how much evaluation your risk appetite demands. A quote that prices "an agent" without counting tools is a guess.

Can a bought agent platform safely use our ERP data?

Only within the permissions you give it. Ask which fields it reads, whether retrieval is scoped by user role, where the data is stored and for how long, and whether any of it reaches a model provider for training. A platform that answers "we handle security" without naming the boundary has not answered.

Where should we start if we want both?

Start with the tool layer and the ledger on a single low-risk task, even if the runtime is temporary. They survive a change of platform, and they tell you within weeks whether the agent is useful or merely impressive.

How do we know it is still working after launch?

Keep a fixed evaluation set and run it before every change, and read the ledger weekly: proposals, approvals, exceptions, and what the exceptions had in common. Activity metrics look healthy while quality drifts; outcome and exception counts do not.