For most of the generative-AI boom, the money chased the models. Bigger context windows, faster inference, cheaper tokens. But a quieter shift is now underway, and it has less to do with what AI can say and far more to do with whether you can trust what it does. As enterprises move from chatbots that answer questions to agents that take actions — booking, refunding, filing, emailing, executing — a new layer of the stack is filling up with capital: the tools that test those agents before they ever touch a live workflow.
It is not glamorous work. There is no demo that makes a room gasp. But the funding flowing into AI evaluation, agent testing, and reliability tooling suggests the market has decided this layer is no longer optional. Here is what is happening, why it matters now, and what teams deploying agents — especially in India — should take from it.
The category emerging
The anchor signal arrived with Patronus AI, which raised a $50M Series B led by Greenfield Partners, bringing its total funding to roughly $70M, according to TechStartups citing TechCrunch. The pitch is deceptively simple: build simulated environments where companies can test how AI agents perform before they are deployed into real workflows. Think of it as a wind tunnel for autonomous software — a place to watch an agent fail safely, repeatedly, and instructively, instead of discovering its failure modes in production with a customer on the other end.
Days earlier, a startup called Probably raised $9M from Andreessen Horowitz, per TechCrunch, with a narrower but related mission: catching hallucinations and factual errors before they reach users. The company is reportedly chasing near-deterministic accuracy — around 99.99% — which is a striking ambition in a field built on probabilistic systems. Whether that figure holds up in the wild is a fair question, but the direction of travel is unmistakable.
Two deals do not make a category, but they rhyme. Evaluation suites (“evals”) and sandboxes are quietly becoming must-haves rather than nice-to-haves. The same way no serious software team ships without a test suite, no serious AI team will soon deploy an agent without a way to measure, simulate, and audit its behaviour first. The trust layer is being funded into existence.
Why it’s needed now
The urgency comes down to a single distinction: agents act, they do not merely answer. A chatbot that hallucinates a wrong fact is embarrassing. An agent that hallucinates a refund, sends the wrong invoice, escalates a ticket to the wrong team, or executes an API call against production data is expensive — and sometimes legally exposed.
When the worst case for a bad AI output was a confused user, you could ship fast and patch later. That calculus breaks the moment AI is wired into autonomous workflows. The failure surface expands dramatically: an agent chains multiple tool calls, makes decisions based on intermediate outputs, and compounds small errors across a sequence. A 95% reliable single step becomes far less reliable across a ten-step workflow. The math is unforgiving, and the costs land in the real world.
This is the structural reason the trust layer is booming now and not eighteen months ago. The industry has crossed from “AI that drafts” to “AI that does,” and the people signing off on these deployments — risk officers, compliance teams, CTOs — want evidence, not vibes. They want to know how the agent behaves under edge cases, adversarial inputs, and ambiguous instructions before it is handed the keys. Evaluation is how you generate that evidence.
What good testing looks like
So what should teams actually expect from this emerging tooling? A few capabilities are becoming table stakes.
- Simulated environments before live deployment. The Patronus approach — staging an agent in a sandbox that mimics the real workflow — lets you observe behaviour without consequences. You can throw thousands of synthetic scenarios at an agent and measure how it handles each, from the routine to the absurd.
- Regression and behaviour checks. Models and prompts change constantly. A change that improves one task can silently break another. Good testing treats agent behaviour like code: every update runs against a suite of known cases, and you get alerted when behaviour drifts. Without this, you are flying blind every time you swap a model version.
- Hallucination and factual-accuracy detection. This is Probably’s lane — catching fabricated facts and unsupported claims before they reach a user. For any agent that surfaces information or makes claims a customer will act on, this check is foundational.
- Human review and audit trails. The best systems do not aim to remove humans; they aim to put humans where they add the most value. That means flagging low-confidence or high-stakes actions for review, and logging every decision an agent makes so that when something goes wrong — and it will — you can trace exactly what happened and why.
The throughline is that reliability is measurable, repeatable, and auditable, or it is not really reliability at all. “It seemed to work in testing” is not a standard any regulated enterprise can stand behind.
The India read
For Indian enterprises, this story is not abstract. Indian companies — across BFSI, IT services, e-commerce, and increasingly the public sector — are deploying agents at speed, often faster than their governance frameworks have matured. The appetite to automate customer support, back-office operations, and internal workflows is enormous, and the cost advantages are real. But speed without a trust layer is how you accumulate risk you cannot see.
The most consequential shift to watch is procurement. As enterprise buyers and Indian IT services firms productise agentic offerings for global clients, testing and evaluation are starting to appear as requirements in the contract, not afterthoughts in the roadmap. A bank will ask how you validated the agent. An enterprise client will ask for the audit trail. “Show me your evals” is becoming a question vendors must answer, much as “show me your security posture” became one a decade ago.
For Indian IT services in particular — firms whose entire value proposition rests on delivering reliable, accountable software at scale — this is an opportunity as much as an obligation. Reliability engineering for AI agents plays directly to the sector’s strengths in process discipline, testing rigour, and managed services. The teams that build evaluation in from day one, rather than bolting it on after a public failure, will win the trust of enterprise buyers who have been burned before by undertested automation.
The practical advice for any operator deploying agents is the same in Bengaluru as in San Francisco: treat the trust layer as part of the build, not a tax on it. Budget for evaluation. Insist on sandboxes and regression checks. Keep humans in the loop for high-stakes actions. And remember that the $50M and $9M flowing into this category are not bets that AI agents will fail — they are bets that the companies who measure their agents will be the ones still standing when the hype settles. The unglamorous work is, as usual, where the real value lives.
