Find out where your agent breaks. Before your customers, your board, or a courtroom does.
A fixed-price, five-day independent audit of one production agent. Calibrated human evaluators. 80+ documented scenarios across the five dimensions where agents actually fail. A 0–100 score you can benchmark, defend, and forward.
Who orders this audit
The stalled deal.
Your champion at the enterprise loves the product. Their security and risk team just sent a questionnaire with a section your dashboards can't answer: "Describe your independent testing regime." A CARA report is a forwardable answer with a score on the cover.
The quiet worry.
You watch the logs. Something's off in maybe 2% of sessions — enough to feel, not enough to reproduce. You built the agent, which is exactly why you'll test for what you built. We won't.
The accountability problem.
You didn't choose this agent. It arrived through a business unit, a vendor, an acquisition. Now it's yours, the board has questions, and "the vendor says it's fine" is not a sentence you're willing to say twice.
The five dimensions
Every audit scores all five. Every finding under each carries reproduction steps and a frequency — evidence, not anecdote.
Task Completion
Does it finish multi-step workflows, or abandon them silently?
WHAT A FINDING LOOKS LIKE: Exits refund workflow at step 3 in 46% of runs; user receives no notice.
Output Consistency
Same input, same quality?
WHAT A FINDING LOOKS LIKE: Three silent behavior changes in 60 days; no version pinning.
Tool Use
Accurate, efficient calls?
WHAT A FINDING LOOKS LIKE: CRM tool over-triggered on 31% of interactions; ~$2,400/mo excess API cost.
Safety & Guardrails
Can it be manipulated?
WHAT A FINDING LOOKS LIKE: Scope bypassed in 2 of 5 adversarial runs; system-prompt fragments leaked.
Eval Infrastructure
Can your team even see what it's doing?
WHAT A FINDING LOOKS LIKE: No regression suite; last three releases shipped unevaluated.
Our evaluators are three people your test suite isn't.
A frustrated customer who wants a refund and won't phrase anything the way your prompts expect. A confused first-timer who takes every instruction literally. A deliberate attacker with time and intent. Every scenario is run by calibrated human experts against the interface your users actually touch — then documented with reproduction steps, so your engineers never have to take our word for anything.
What lands on day 5
One number your board understands, computed across five weighted dimensions.
Severity, reproduction steps, frequency across runs, and business impact — your engineers never take our word for anything.
Fixes prioritized by risk-reduction per engineering hour, specified so your own team can execute.
Two pages, business language, written to be forwarded upward — or to a customer.
Findings walked through with your team; hard questions welcome.
Your score placed against the CARA benchmark for your vertical — context, not just a number.
The week, hour by hour
- DAY 0Intake call
45 minutes. Scope, workflows that matter, access to the same interface your users touch.
- DAY 1–2Scenario battery
80+ scenarios run and documented across all five dimensions.
- DAY 3Adversarial pass
Deliberate manipulation, edge cases, and business-logic probing.
- DAY 4Verification
Every finding independently reproduced by a second evaluator.
- DAY 5Report delivery
Score, findings, roadmap, executive summary — in your inbox.
- DAY 6Live readout
60 minutes with your team. Hard questions welcome.
TOTAL TIME REQUIRED FROM YOUR TEAM: ~2 HOURS
Auditing an agent you bought, not built?
Same audit, two additions: we coordinate scope with your vendor under NDA (or test entirely black-box — their cooperation is helpful, not required), and the executive report is written for the people you answer to: business impact first, remediation responsibility mapped to vendor vs. in-house. If you're pre-contract rather than post-deployment, you want CARA for Procurement →
Before you say yes
The questions sharp buyers ask — answered in daylight, hardest first.
Keep them — we integrate nothing and replace nothing. Tools measure what's deterministic and what your team anticipated. We find adversarial behavior, business-logic gaps, and real-user failure patterns — and we produce something no internal tool can: independent evidence.
MIT's enterprise data: externally built and evaluated tools succeed about twice as often. The people who built the agent test for what they built. That's not a criticism. It's the reason audit firms exist in every other discipline.
Founding clients contribute anonymized results to the CARA benchmark — the industry baseline we publish. You're compensated for that contribution with the discount. Spots are limited; the price steps up on a published schedule as results ship.
50% of the fee back, per the guarantee. It has never happened. We publish the day it does.
No. Behavioral testing only, through the user-facing interface. Mutual NDA first, on request. EU-based evaluation available for EU clients.
You. Findings feed the benchmark only as anonymized, aggregated statistics — never identifiable, never shared, never used in our marketing without written permission.
We think about this more than our clients do. Findings are delivered before any proposal. The report is yours outright. Remediation through Ingenize is optional, always disclosed, and never required to act on the roadmap — every fix is specified so your own team can execute it.
Customer-support, internal-ops, fintech, legal, health, HR, code assistants, data pipelines, multi-agent systems. If it takes actions or produces output for real users, we can audit it.
Intake call within 48 hours of the strategy call; report five business days after intake. Post-incident rapid engagements available — ask.
Bundled at list-minus-20% per additional agent; three or more is a custom package. Scope it on the free call.