Questions to Ask an AI Agent Vendor Before You Buy


Before you sign with an AI agent vendor, ask five questions: who approves what goes live, how autonomy is raised over time, what your agents read from your own knowledge base, what each action costs and where spending is capped, and what the audit record shows after the fact. Get clear answers to all five, then run a bounded pilot on one real outcome. Everything else is a demo.
Why do most AI agent vendor evaluations fail?

Most teams evaluate on features, not on accountability, and they regret it.
A vendor demo shows an agent writing copy, filing a brief, and posting to a channel. It looks fast. It looks smart. What the demo never shows is what happens when the agent posts something off-brand at 2 AM, who stopped it, and what the record says afterward. That gap is where brand-safe content teams get burned.
The five questions below are designed to close that gap. They are not trick questions. A vendor with a serious product answers them directly. A vendor who deflects or reframes is telling you something important.
Who approves what goes live?

Every agent action that touches a customer-facing channel needs a clear approval path before you deploy.
Ask the vendor: can you show me, in the product, exactly where a human approves a post, a campaign asset, or an outreach sequence before it publishes? Not in theory. In the product.
On Nagent, every action is proposed, approved, run, and audited. Agents start supervised. Publishing waits for a named person at every level below the top one, and deleting or governance changes always require a person. That is not a policy document. It is how the control plane works.
If a vendor talks about configurable approvals, ask them to show you the default. The default tells you what they built the system to do.
How does agent autonomy increase over time?

Autonomy should be earned on a track record, not switched on at setup.
This question separates platforms built for brand safety from platforms built for speed demos. The right answer is that an agent starts at the lowest autonomy level, proposes actions, and is promoted only when its track record justifies it.
Nagent uses a five-level autonomy structure: L1 observe, L2 assist, L3 operate, L4 represent, and L5 lead. Agents earn their level; every agent arrives at L2, and none starts at the top. The platform drops a level when an agent underperforms. Promotion happens on evidence, one level at a time, and a person decides it.
Ask your vendor: what is the lowest autonomy level, what does an agent do at that level, and what evidence triggers a promotion? If the answer is a slider or a checkbox, the system has no real accountability model.
What do agents read from your own knowledge base?
An agent is only as brand-safe as the knowledge it works from.
A generic agent trained on public data will produce generic, and sometimes wrong, output. A brand-safe agent reads your tone guidelines, your approved messaging, your product catalog, your campaign history. Ask the vendor how your proprietary knowledge gets into the agent, who controls it, and whether it stays isolated from other customers' data.
Nagent agents work from a shared context that includes your team's own knowledge. Tenant isolation is a platform capability, not an add-on. On the Enterprise plan, deployment happens inside your private VPC.
The practical test: give the vendor a one-page brand guideline and ask them to show you an agent output that reflects it. If the output looks generic, the knowledge layer is not working.
What does each agent action cost, and where is spending capped?
Uncapped agent spending is a budget risk, not just a financial one. It signals a platform built without accountability controls.
Every agent action should carry a cost in credits or compute, and that cost should be visible before the action runs. You need a daily cap, a monthly cap, and a kill switch that stops all activity immediately if something goes wrong.
Nagent agents carry both a daily and a monthly spending cap, set in credits. The platform has a kill switch. Budget caps are a confirmed platform capability. On the Business plan, credits are pooled across the team.
Ask the vendor to show you the cap configuration screen. Ask what happens when the cap is hit: does the agent stop, alert, or queue? Ask how long the kill switch takes to propagate. These are operational questions, not edge cases.
What does the audit record show, and who can see it?
If you cannot read exactly what an agent did and why, you cannot defend a brand decision made by that agent.
The audit record is where accountability lives. It should be append-only, meaning no one can edit or delete it after the fact. It should show the action, the model that ran it, the approval that authorized it, and the outcome. It should be accessible to your compliance and legal teams without a support ticket.
Nagent maintains an append-only audit record for every agent action. A decision ledger is a confirmed platform capability. Role-based access control determines who sees what inside the workspace.
Ask the vendor to show you a real audit record from a live environment. Ask whether it is append-only. Ask who can access it and how. A vendor who claims comprehensive logging but cannot show you the actual record is not answering the question.
How do you run a pilot that actually tests brand safety?
Pick one real outcome, one real channel, and one real piece of content. Run it with approval gates on.
A pilot on a synthetic task tells you nothing. A pilot on a real campaign asset, reviewed by your brand team, approved through the platform's actual approval flow, and published to a live channel, tells you everything.
Structure the pilot around three checkpoints:
Does the agent's first draft reflect your brand guidelines without manual correction?
Does the approval flow catch and hold anything that should not go live?
Does the audit record show you what happened, in plain language, after the fact?
If all three pass, you have a vendor worth expanding. If one fails, you know exactly what to fix before you scale.
Nagent supports evaluation before release as a platform capability, and where the plan includes it, a Nagent marketer can join your team and own the pilot outcome with you.
What does a complete vendor checklist look like?
Use these five questions as a structured checklist, not a conversation starter.
Print this and bring it to the demo:
Approval gates: Show me, in the product, where a human approves before anything goes live.
Autonomy model: What is the lowest autonomy level, and what triggers a promotion?
Knowledge layer: How do my brand guidelines get into the agent, and are they isolated from other customers?
Spending controls: Show me the daily cap, the monthly cap, and the kill switch.
Audit record: Show me a real audit record. Is it append-only? Who can access it?
Then run the pilot. One outcome. One channel. Approval gates on.
A vendor who answers all five clearly and runs a clean pilot has earned the next conversation. A vendor who deflects on any of them has answered the most important question of all.
Related reading
Frequently Asked Questions
What is the single most important ai agent vendor evaluation question?
Ask who approves what goes live, and then ask the vendor to show you that approval flow inside the actual product. A platform that defaults to autonomous publishing without a human gate is not built for brand-safe content. Everything else on the checklist follows from this one.
How do I know if an AI agent's autonomy model is safe for brand content?
A safe autonomy model starts agents at the lowest level, where they can only suggest, and promotes them level by level based on a verified track record. Platforms that let you set autonomy with a slider or a checkbox have no real accountability structure. Ask the vendor what evidence triggers a promotion and what happens when an agent underperforms.
What should an AI agent audit record include?
A complete audit record shows the action taken, the model that ran it, the approval that authorized it, and the outcome, and it must be append-only so no one can alter it after the fact. Role-based access control should let your compliance team read the record without needing a support ticket. If a vendor cannot show you a live audit record during the demo, treat that as a red flag.
How do I structure an AI agent pilot for a D2C brand?
Pick one real campaign asset, run it through the platform's actual approval flow, and publish it to a live channel. Check three things: whether the first draft reflects your brand guidelines, whether the approval gate holds anything off-brand, and whether the audit record explains what happened in plain language. A synthetic pilot on a fake task tells you nothing useful about brand safety.
What should you do next?
Book a demo with Nagent and bring the five-question checklist. We will walk you through each answer in the live product, not a slide deck.