pCloudy

The AI Quality Infrastructure Benchmark Report 2026

We surveyed 100+ QA and engineering leaders across banking, technology, healthcare, hospitality, media and logistics on how they actually build AI-native testing — the tools, the models, the context layer, and the compliance walls they keep hitting.

Get the full benchmark report

See where AI testing is actually heading in 2026 — what 100+ QA and engineering leaders are adopting, dropping and betting on next.

No spam. We'll only email you about this report.

100+QA & engineering leaders
40%Director-level or above
7Industries covered
Read the findings Benchmark your team
Tooling reality
6 in 10

pair open-source frameworks with AI writing assistance

Compliance wall
8 in 10

are prohibited or significantly restricted from sending test data to a public AI API

Execution gap
5 in 10

execute under 70% of their suite in a release window

Ship confidence
9 in 10

are confident at ship — even though 45% leave 30%+ of the suite unrun

Chapter 01

AI solved writing. Not running.

AI has not replaced the framework — it has been bolted onto it. The suites underneath are still mid-sized, slow, and only partially executed before ship.

6 in 10
Hybrid approach

Test creation has gone hybrid, not autonomous

59% pair an open-source framework with an AI assistant rather than buying an all-in-one AI test generator. Only 10% run a commercial AI test generation tool in production, and 14% remain mostly manual.

Primary test creation approach
Open-source frameworks + AI assistance59%
Mostly manual14%
Commercial AI test generation (in use)10%
Open-source frameworks, no AI10%
Commercial AI test generation (evaluating)7%
7 in 10
Use Playwright

The stack is assembled, not bought

Playwright leads at 66%, and generic chat assistants — Copilot, ChatGPT — sit level with Appium at 48%, ahead of Selenium at 41%. Purpose-built AI testing tools trail: the AI layer is added to whatever is already approved for developers.

Frameworks and AI tools in use
Playwright66%
REST Assured / Postman / Newman55%
Appium48%
Selenium / WebDriver41%
Copilot / ChatGPT for test writing48%
QPilot (Pcloudy)14%
Commercial AI testing tools (other)14%
XCUITest / Espresso7%
5 in 10
Run 500–2,000 cases

Mid-size suites dominate

45% sit in the 500–2,000 band per release cycle — large enough to be slow, small enough that nobody has industrialised maintenance. Only 7% run more than 10,000 cases.

Test cases per release cycle
500–2,00045%
Fewer than 50024%
2,000–10,00021%
More than 10,0007%
We do not track this3%
5 in 10
Ship under 70% executed

The execution bottleneck nobody is pricing in

45% ship with less than 70% of their automated suite executed, and 17% get under 40% through. Only 14% clear more than 90%. The coverage gap is an infrastructure and throughput problem, not a test-writing problem.

Share of the automated suite that runs before ship
70–90%34%
40–70%28%
More than 90%14%
Less than 40%17%
We have not measured this7%

“Get the QA process fundamentals right first. AI can accelerate a good process, but it cannot fix a broken one.”

George Ukkuru · Practice Director, Speridian Technologies
Chapter 02

Policy says one thing. Practice says another.

Model choice has quietly become a governance decision. On policy, 8 in 10 teams are prohibited or restricted from sending test data to public AI APIs. On deployment, a separate question, 76% still touch a public frontier model — 45% straight through a direct API.

8 in 10
Blocked or restricted

Compliance sets the agenda — practice hasn’t caught up

46% say sending test data to a public AI API is outright prohibited and a further 33% operate under significant restrictions. Only 4% report no restriction at all — yet nearly half still route workloads through a public frontier model.

Constraints on sending test data to public AI APIs
Yes — prohibited46%
Yes — with significant restrictions33%
Partially — varies by team or project17%
No restrictions / not formally assessed4%
76%
Still touch a public model

The deployment mix doesn’t match the stated priority

45% go direct to public frontier models via API with no gateway or DLP layer in between, 17% mix public and local by use case, and 14% route through a managed gateway. Only 7% run fully self-hosted open-source models.

Where QA models actually run
Public frontier models via API45%
A mix of public and locally deployed17%
Public frontier via a managed gateway14%
Open-source models on own infrastructure7%
Do not use LLMs in QA10%
Evaluating, not yet chosen7%
7 in 10
Chose for sovereignty

Compliance now outranks capability

Data sovereignty is the single biggest driver at 67%, ahead of security policy (58%) and raw model performance (54%). Cost barely registers at 25%, and 8% never made a deliberate choice at all — they use whatever the vendor ships.

Drivers behind public API vs local deployment
Data sovereignty and compliance67%
Security policy58%
Performance and capability54%
Cost25%
Simplicity of integration21%
No deliberate choice — vendor default8%
7 in 10
Rely on prompts

Context management is the unbuilt layer

Prompts are still the primary memory: 71% rely on prompt and system-prompt engineering, 50% have added RAG, and only 25% maintain structured memory across sessions. Retrieval gives a model documents to look up, but not a record of what it learned last time — so quality still resets between sessions.

The knowledge layer behind the model
Prompt engineering and system prompts71%
Retrieval-Augmented Generation (RAG)50%
Fine-tuned or domain-adapted models33%
Structured memory across sessions25%
None — immediate input only4%

“Never use AI for autonomous execution — always keep a human in the loop so humans stay in control.”

Sandeep Garud · CEO, SpurQLabs
The full report · PDF

Take the whole benchmark with you

The complete 2026 data set — tooling, model deployment, context, agents and compliance — with every chart and cut.

Download the report Email only · instant access
Chapter 03

The maintenance tax and the agent gap

The clearest pattern in the data: teams adopted AI faster than they built the context and memory infrastructure it needs to stop repeating itself.

5 in 10
Spend 10–20% on upkeep

The maintenance tax

Flaky tests, broken locators and outdated scripts eat real capacity. 54% spend 10–20% of a release cycle on upkeep, and 12% burn more than a third of every cycle on the same firefighting — for a 10-person QA org that is three engineers doing nothing but repair work.

QA time spent on test maintenance
10–20%54%
Less than 10%21%
21–35%13%
More than 35%12%
We don’t track this0%
3 in 10
Multi-agent orchestration

AI assists individuals, not the system

34% use AI with no agentic pattern and 29% run one scoped agent; nobody is outside AI entirely. Of the 37% running multiple agents, 29% coordinate with shared state — the other 8% are fixed pipelines passing output down a chain.

How AI is wired into the process
AI tools, but not an agentic pattern34%
Multi-agent orchestration (shared state)29%
Single agent for a specific task29%
Multiple agents in a fixed handoff or pipeline8%
Do not use AI agents in testing0%
66%

of teams use Playwright — the most adopted framework in the study

1 in 4

teams lose over a fifth of every release cycle to test upkeep

25%

keep structured memory across sessions; most of the rest lean on retrieval or prompts that carry nothing forward

“AI can generate tests, but transforming undocumented legacy back-end applications into reliable, maintainable regression suites with realistic test data remains a significant challenge.”

Mohamed Vaquas Umair · VP, Head of Engineering, Velocix India Solutions
Chapter 04

Confidence is high. Coverage is not.

93% are confident in release quality at the point it ships — while only 48% get more than 70% of their suite executed. That gap is the report’s sharpest finding, and it sits on infrastructure that is still mostly owned, racked and maintained in-house.

9 in 10
Confident at ship

Nobody is unconfident — and that is the problem

45% are extremely confident and 48% somewhat confident at the point a release goes to production. Not one respondent said they were unconfident. Set that against Chapter 01: 45% ship having left 30% or more of their automated suite unrun. Confidence is being derived from something other than executed coverage.

Confidence in release quality at production
Extremely confident45%
Somewhat confident48%
Uncertain7%
Somewhat unconfident / not confident0%
4 in 10
Own an in-house lab

Execution still runs on owned hardware

38% still operate a physical in-house device lab, 34% are on single-tenant private cloud and 31% use a public cloud testing platform. A further 17% run an on-prem device cloud behind the firewall — the compliance posture from Chapter 02 showing up as racked hardware. Just 10% test on emulators alone.

Primary execution infrastructure
Physical in-house device lab38%
Private cloud — single tenant34%
Public cloud testing platform31%
Self-managed CI/CD grid31%
On-prem device cloud behind the firewall17%
Emulators or simulators only10%

“Trust in execution results, not test creation. AI has made authoring fast and scalable, but we lack equally mature AI-assisted triage to reliably separate real defects from environment noise — so confidence hasn’t scaled with volume the way authoring has.”

Sankaranandhan Sundaresan · Digital Engineering Lead, MGM Resorts International
Chapter 05

Where the next 12 months of budget goes

Asked what they will invest in next, teams did not pick more test generation first. They picked making the agents they already have reliable — and the open-text answers say exactly why.

7 in 10
Agent orchestration first

Reliability beats generation

69% name agent orchestration and context management as an investment priority — ahead of AI test generation at 59%. Observability follows at 41% and execution infrastructure at 34%. Only 14% plan no significant investment. The market has moved from “can AI write tests” to “can we trust what it produced”.

QA infrastructure investment, next 12 months
Agent orchestration & context management69%
AI test generation59%
Test observability and failure analysis41%
Execution infrastructure34%
Local or private AI model deployment21%
No significant investment planned14%
6 in 10
Cite context or trust

The blocker, in their words

Asked for their single biggest unresolved challenge with AI in testing, respondents wrote answers that cluster into four themes — and almost none of them are about generating tests. Domain context and trust in results dominate.

Share of open-text answers
Missing business or domain context32%
Trusting results — flakiness, triage, validation28%
Prompting and model-selection skill gap16%
Cost, tokens and approval friction16%
No blocker reported8%
69%

will invest in agent orchestration and context management next

93%

confident at ship — while under half execute 70%+ of the suite

38%

still run tests on an in-house physical device lab

“We’re redefining the STLC through agentic AI, accelerating releases while making QA smarter, more efficient, and more impactful.”

Priyanka Ajith · QA Manager, Tricog Health India
Contributors

Voices from the field

Beyond the survey, QA and engineering leaders told us in their own words where AI is working, where it is not, and what they are building next.

“We treat Generative AI as a hyper-efficient assistant that does 80% of the heavy lifting in seconds, while relying on human critical thinking for the 20% that actually guarantees quality.”

Jayprakash MouryaAVP · DBS

“AI now spans our whole QA loop — authoring the plan, driving the browser, healing broken locators, publishing results, and keeping the suites green in CI.”

Archana BangarLead QA Automation Engineer · Blend

“AI is helping improve test coverage and accelerate debugging. We are looking forward to leveraging agent orchestration to autonomously execute tests, reducing manual effort and minimizing the need for human intervention.”

Sarala PonnagantiLead Software QA Engineer · Suncoast Credit Union

“Looking to improve leaps and bounds in using AI-powered QA, demonstrating cost reduction by improving release quality and cutting down rework.”

Giridharan Kalpathi SukavaneswaranHead Digital QA and Banking, ME & KSA · Horváth Partners

“We’re redefining the STLC through agentic AI, accelerating releases while making QA smarter, more efficient, and more impactful.”

Priyanka AjithQA Manager · Tricog Health India

“AI industry focus is more on scripting, while analysis and validation still lack a lot.”

Mohamed AbdelkaderSenior QA Manager · MBC Group
01 / 06
Interactive

Check how release ready is your organization

Answer as your team actually operates, not as the roadmap says. Each answer shows how many of the 100+ respondents sit where you do.

01

How does your team primarily create tests today?

02

What share of your automated suite runs inside a standard release window?

03

How much of a release cycle goes to maintaining existing automation?

04

How are AI models deployed in your QA workflow?

Scored on governed access, not on where the weights live: a managed gateway, a deliberate public/local split and self-hosted open source all count as controlled paths. Only ungoverned direct API calls score lower.

05

How does your team manage context for AI in testing?

06

How does your team use AI agents in testing?

Move the bottleneck

The data points one way: the constraint is execution throughput and context, not test writing. Let’s look at where your release window actually goes.

Download the report Talk to our team Explore the device cloud