Dinakar
All right, great.
Dinakar
Hello everybody and welcome to Pcloudy webinar on the QA leaders guide to building an agentic quality platform.
Dinakar
My name is Dinakar and I will be the host for today.
Dinakar
Before we get started, I would like to go over a few housekeeping rules.
Dinakar
Uh you may have noticed that your line is currently muted.
Dinakar
And if you do have any questions, you can submit them using the Q&A option given at the bottom of the screen.
Dinakar
Uh we will answer your questions towards the end of the webinar in the Q&A section.
Dinakar
Please note that this webinar is being recorded as well and we will send out the link of the recording to share with your colleagues later or watch it a second time.
Dinakar
With that said, let me introduce the speaker for today, Avinash Tiwari.
Dinakar
Avinash is a renowned thought leader, a recognized keynote speaker and also the co-founder of Pcloudy.
Dinakar
With over two plus two decades of experience in product development and testing, Avinash brings with him a passion for AI and other emerging technologies.
Dinakar
He is a firm believer of quick adoption and agility in the tech space.
Dinakar
All of which have solidified his reputation as a leader in the world of digital testing.
Dinakar
So, without further ado, let me hand it to our speaker, Avinash Tiwari.
Dinakar
Over to you, Avinash.
Avinash
Uh hello everyone.
Avinash
And thanks Dinakar for the warm introduction.
Avinash
Um I'm super excited to host all of you here for this very special topic um around building agentic AI platform.
Avinash
I mean, at Pcloudy, we have been using AI in various forms for more than 5 years now.
Avinash
But about 2 years ago, uh when gen AI burst into our scene, the excitement reached an entirely new level.
Avinash
And since then, every day has brought fresh curiosity about the future and also a lot of uh excitement around how it will transform our world of quality engineering.
Avinash
And I'm sure you all must be uh feeling the same way as we have seen the progress in last 2 years.
Avinash
So, at this moment, uh I truly believe that we are at the juncture of one of the most uh exciting or to be more precise, unique time in the history of technology.
Avinash
And the transformation that we are seeing isn't just coming, I feel it's already here.
Avinash
So, today when we are going to talk about uh the the agentic AI in quality engineering, uh I I firmly believe that it's going to change a lot of perspective in the way uh you think about quality engineering and some of you might be already uh thinking in that direction.
Avinash
So, I would love to hear your thoughts and we can keep it interactive, so you can share your thoughts in the Q&A session.
Avinash
I would love to know more about your journey and how you envision agentic AI in the quality engineering space.
Avinash
So, before we get into the core of uh today's discussion and agenda, uh I want to start with a quick experiment or maybe a thought exercise just to understand a point of view from the perspective of how fast things are moving.
Avinash
So, what I would like you to do is uh and you can cheat because we have we can't see each other.
Avinash
Or you can open chat GPT and answer this question.
Avinash
Um it's a quick thought exercise where you close your eyes and think about breakthroughs that you have heard or you came across uh in the world of AI in last 30 days.
Avinash
And I'm not talking about uh quarter or maybe last year or last 2 years.
Avinash
I'm talking about just last 30 days.
Avinash
So, just go back to your thoughts and try to figure out what all you heard about uh kind of announcements or breakthroughs in the world of AI.
Avinash
Great.
Avinash
I mean, you can put your thoughts in chat how many uh announcements that you could think about.
Avinash
Uh and I'm sure you must have been surprised uh with the kind of thoughts that you might have come across.
Avinash
Some of you lost count.
Avinash
Uh I definitely uh lost my count.
Avinash
And to give you a sense, I'm not sure you all attended the open AI uh dev day summit that happened last uh night.
Avinash
And uh if you're aware of those announcements, so open AI launched uh app SDK, which means that you can access any app are using their app SDK directly from chat GPT.
Avinash
And I have been thinking about interfaceless UI uh UI-less kind of application and with this announcement, I think we are moving to that direction where application UI probably in some years might become meaningless.
Avinash
You can access everything uh through a prompt and perform all the tasks through a prompt.
Avinash
Why do you need UI navigation?
Avinash
They also announced the agentic AI platform where you through prompt you can build a whole agent orchestration.
Avinash
Something which uh which earlier would have taken so much time for anyone to even think and build and uh bring it to to life to be used in production kind of systems.
Avinash
And if you can go back maybe 15 days, 20 days I mean, earlier, Cloud Cloud had announced a similar kind of features, especially with their new Cloud Cloud Sonnet model.
Avinash
You can think on and on about the number of announcements that we came across.
Avinash
Another big one was open AI and video partnership to invest 100 billion in AI infrastructure.
Avinash
So, just imagine the scale at which AI will get deployed into various domains.
Avinash
And that's exactly is the point.
Avinash
The the pace of AI change isn't just fast, it's truly overwhelming.
Avinash
And sometimes it's it's dizzy uh to catch up with what's happening in the world of AI and the way things are changing around us.
Avinash
And as we speak, it's accelerating.
Avinash
In certain areas, I also see that the capabilities are plateauing, but still to catch up with what has been in front of us and the way some of the new developments are happening, it's still overwhelming for all of us.
Avinash
Now, having gone through that thought experience thought ex- exercise, um I want to kind of talk a little bit bit about why AI as a change is different from what we have seen uh as technology waves in past.
Avinash
And I call it a leverage equation.
Avinash
So, if we I mean, I'm sure we all have experienced different technology waves in our career last couple of decades that I have seen I have seen multiple waves.
Avinash
But in all other waves, uh the outcome or the productivity gains from those technology changes were typically uh linear, right?
Avinash
So, if you bring a new tool, you will see some productivity gain in terms of let's say 20%, 30%, 50%, even maybe double, 100%.
Avinash
But with the AI, I think the gains are becoming exponential.
Avinash
And that is one big reason why the this wave is very different.
Avinash
And we are already seeing, for example, uh we have seen support teams getting completely revolutionized with the AI-driven chatbots.
Avinash
The marketing teams are able to handle maybe 50 campaigns which earlier used to take five people, but maybe one or two people are uh right now enough to do that kind of work.
Avinash
Development teams are already seeing increase in velocity in terms of code generation.
Avinash
We are seeing the same in in in case of uh lot of other areas.
Avinash
And uh that's where I mean, in the slide, I have put a question mark on the quality engineering and that's the answer we would like to have today.
Avinash
Um what does exponential leverage look like in the space of quality engineering?
Avinash
Right?
Avinash
And what's the new paradigm here?
Avinash
What is making it possible, right?
Avinash
I mean, we talked about leverage equation, but if we just kind of drill a little bit deeper, uh what are the changes which are bringing this kind of exponential gain uh to the the to the world of development and a lot of other areas, uh, which are impacted here.
Avinash
And I've listed three things.
Avinash
There are many, uh, impact, uh, which is causing this exponential, uh, uh, improvement in the productivity, but I've listed three that I have come come across and it is impacting even my work and how I have been able to become more productive with the help of AI.
Avinash
So, first, English has become a new programming language.
Avinash
So, you are not supposed to write maybe test scripts anymore, right?
Avinash
You describe intentions and AI is able to provide the answer to you.
Avinash
And you can look at this as a threat, but after a long time there is a technology which is helping everyone become a creator.
Avinash
So, we are not just executors anymore.
Avinash
We all can become creator because English has become the new programming language.
Avinash
Now, at today when I have to go and tell my developers to develop a particular feature, I create a small prototype using cursor or lovable, I mean, combination of lovable and cursor and show it to them that this is what I'm looking at.
Avinash
Can you build it for me?
Avinash
Earlier, I had to either explain them in in form of a verbal, uh, discussion or create a kind of document, but now I can small prototype, working prototype, and show it to them.
Avinash
That's a change.
Avinash
And we all can become creator with this huge leverage that we have that English is the new programming language.
Avinash
A huge shift for all of us.
Avinash
And we we should really, uh, adapt to this change rather than being threatened by it.
Avinash
And that is kind of leading to this whole vibe-based coding and testing.
Avinash
I'm not saying that this is, uh, an end uh, in terms of that vibe coding or vibe testing can do everything, but it can immensely improve the way we work right now or the way we could the way we can become productive in the future.
Avinash
And third, related to this is it's helping us in terms of evolving roles and expectations.
Avinash
So, like I said as part of the first point that we all are becoming creators.
Avinash
So, from doers to creators to to strategist.
Avinash
That's the level of evolution which is happening in terms of roles and expectation across the board and including QE.
Avinash
Uh, and that's one big impact which we are seeing because of this whole paradigm shift with the AI revolution.
Avinash
So, And and just to before we talk about QE, uh, it's not just about the area of development and testing which is getting changed.
Avinash
Every single industry is being impacted.
Avinash
Right?
Avinash
So, if you look at legal, now AI can read thousands of contracts per hour and it can summarize your contracts, find out uh, discrepancies within seconds and minutes, which used to take so many hours for a particular lawyer to go through and figure out.
Avinash
In health care, we are seeing AI diagnosis improving the accuracy and supporting doctors uh, and and saving lives.
Avinash
In finance, now we can detect frauds in milliseconds.
Avinash
In retail, you can predict the demand with uncanny precision.
Avinash
In in, uh, entertainment industry, now you can generate videos or even movies, uh, with the help of AI.
Avinash
You don't need a huge production infrastructure to do that.
Avinash
So, everyone can become a movie creator now.
Avinash
And that is true for even, uh, um, quality engineering space, right?
Avinash
Um, so we can't afford to be in past where we are still writing test cases manually or writing brittle test scripts, struggling with your test coverage.
Avinash
The question is isn't if the quality engineering will be transformed by AI, but how fast.
Avinash
And we are already seeing that change.
Avinash
I need not kind of repeat that statement that, uh, the change is coming.
Avinash
The AI revolution or we are already seeing testing is one of the areas which are getting impacted faster than some of the other areas in the space of, uh, uh, application development.
Avinash
Now, how how have we reached there?
Avinash
Just to trace the evolution, uh, over last 20 years and this is based on my experience over last two decades.
Avinash
Um, very quickly, era one was manual where we were writing scripts or we were not writing scripts, we were writing test cases in a spreadsheet.
Avinash
Human execution, take screenshots, log bugs.
Avinash
The release cycle was weeks or months per test cycle.
Avinash
I mean, it was traditionally waterfall kind of model that we all, uh, worked in that era.
Avinash
Era two brought auto automation, right?
Avinash
So, we saw evolution of scripts, frameworks.
Avinash
We can we could schedule runs.
Avinash
So, we got down from months to weeks or days depending on the complexity of the applications.
Avinash
We also saw evolution of agile and, uh, devops methodologies.
Avinash
Uh, era third came AI assistance where we saw some tools which could help us in script creation, self-healing, uh, automated executions, etc.
Avinash
Um, typically the devops continuous testing kind of, uh, era I'm talking about where we were able to reduce the overall lead cycles from weeks to maybe days or hours, in some cases even minutes per cycle.
Avinash
We saw organizations doing maybe tens of releases per day.
Avinash
Uh, companies like Facebook, uh, Amazon, they do hundreds of, uh, releases uh, every day and we saw that in this era.
Avinash
But now we are into this era of agentic, uh, quality engineering, right?
Avinash
It is it's not just about so all three eras, if you if you just notice the trend, it was about improving the speed, right?
Avinash
We we saw huge improvement in the speed, uh, at which we could work or, uh, we could release the applications.
Avinash
Era four, which is agentic, is fundamentally different, right?
Avinash
We are talking about autonomous systems.
Avinash
We are self-evolving systems, um, where the quality can I mean, continuous quality can happen without human intervention, right?
Avinash
And whether humans are, uh, supposed to perform a strategist or a orchestration role rather than typical execution role.
Avinash
So, this is not about bringing new tools.
Avinash
This is about fundamentally changing how quality engineering is is, uh, looked at.
Avinash
So, with that as a context of how things are moving so fast, how we have reached, uh, this stage, um, I just wanted to quickly touch upon the impact of this change in the area of testing from some data perspective, right?
Avinash
So, there are two graphs here.
Avinash
One, uh, there is a survey in which it was asked what is the plan to implement AI technology for application development.
Avinash
And you can see the two areas where, uh, everyone said you would like to adopt is in the area of testing.
Avinash
Application test automation and generation of test cases from business requirement.
Avinash
Uh, apart from some of the other like code completion, code suggestion, etc.
Avinash
The two big areas where AI is being adopted is in the space of testing.
Avinash
And similarly, the second, uh, graph, uh, which is what are the what is the plan to, uh, enable the use cases via AI-powered test automation.
Avinash
So, within that also you can see generation of automation automated functional test cases is at the top.
Avinash
Then you we have other areas like accessibility tests.
Avinash
So, the point here is that, uh, if you have not thought about it, this is a low-hanging area where AI can really start making a huge productivity gain for the teams.
Avinash
Um, so, this is the right time to kind of start thinking about the areas where you can start using AI and and make changes for your organization.
Avinash
Yeah.
Avinash
So, uh, with that, I would, uh, now move into the specifics of agentic AI and the change that we are seeing in the world of quality engineering and what should be an ideal agentic AI platform.
Avinash
And to get to that, a quick understanding of what is agentic AI.
Avinash
Right?
Avinash
We have been talking about, um, uh, AI-based systems, gen gen AI, but what is this agentic AI and how is it different from the traditional AI that we have seen, right?
Avinash
Especially for example, we have been using ChatGPT, Claude, or other chatbots for for almost more than two years.
Avinash
Um, so how is it different from what we have been doing so far?
Avinash
So, I'm calling traditional AI as the typical agent that we have seen in form of chatbots or chat GPT or cloud or any other systems that you might be involved with.
Avinash
So, traditional AI uh, is reactive and it waits for command, right?
Avinash
So, for example, you can say "Test the login page." You are giving a command and that traditional AI will execute that command.
Avinash
But, agentic AI is goal-oriented.
Avinash
So, when you say that "Test the login page." or the better way to say uh, to agentic AI platform is "Ensure success of my login uh, screen." So, it will figure out all different possibilities of uh, kind of testing has to be done and perform It can also perform depending on how you how you envision the agentic AI for that particular case.
Avinash
But, point is that agentic AI is typically goal-oriented.
Avinash
The second difference is uh, traditional AI works in isolation, right?
Avinash
Where like if you give the same command "Test login." it will go and execute a login and after that it will just stop.
Avinash
While in case of agentic AI, it can collaborate with other agents, share the information from one particular step to other step so that you can create a much larger You can achieve a much larger objective and to be very frank, the same statement that I made, it can achieve a goal.
Avinash
And to achieve a goal, you need that sharing across different agents and to be able to perform a bigger task.
Avinash
Um, again, traditional AI follows a static rule and gives you an outcome, while agentic AI learns and evolves with uh, the the outcome that it is seeing uh, based on the command that you give it to the agent.
Avinash
So, in a sense, agentic AI e- truly gives you ability to create autonomous systems where it can perceive what kind of environment it is in, it can make decisions, and then it can take the actions to achieve objectives, right?
Avinash
So, it it is in true sense uh, able to mimic uh, human and in our case, quality engineer, rather than focus on a particular task.
Avinash
Right?
Avinash
So, that way, the scope and the uh, goal of our agentic AI system could be at a much bigger scale than a traditional AI system.
Avinash
Now, just going a little bit deeper into the architecture part of it, uh, it has If you look at any agentic AI system, uh, under the hood, the general architecture has these four layers that's that you see on the screen.
Avinash
At the top, we have our orchestration layer.
Avinash
So, think of it as a decision center that coordinates everything, right?
Avinash
So, for example, when you say that "Ensure success of my login page." or uh, "Make sure that my login page works correctly." uh, it will figure out what are different decisions that I need to take.
Avinash
What all coordination do I need to perform, right?
Avinash
So, orchestration layer takes care of all the uh, decision that it needs to perform, right?
Avinash
Uh, then we have the second layer and second layer is divided into three parts.
Avinash
Again, there are various design patterns, but this is one of the uh, simplistic way of representing the overall architecture.
Avinash
So, it it is broken into three parts, sense, think, and act.
Avinash
So, sense means what is happening, right?
Avinash
So, system can I mean, the the agentic AI system can perceive environment through monitoring.
Avinash
And let's take a case of a quality engineering case where we design a self-healing system, right?
Avinash
So, for self-healing system to work, there has to be some someone monitoring where the failure happens, right?
Avinash
So, sense means it's always looking at okay, whether the failure happened or not, right?
Avinash
Once the failure happens, it thinks, okay, if the failure is a genuine failure, do I need to act on it?
Avinash
So, it has its own way of reasoning and then if it thinks that I need to make some changes to the self-healing as to the script to self-heal, then it will plan, okay, what do I need to do?
Avinash
Where do I need to make the change?
Avinash
And then finally act.
Avinash
So, and then it decides where do I need to go?
Avinash
I mean, what all changes do I need to need to make?
Avinash
And then it acts.
Avinash
And then this is not a sequential activity, sense, think, and act.
Avinash
It it goes in a loop.
Avinash
So, after act, let's say it will figure out whether it's now correct or not.
Avinash
And within that process, it might go and think again, right?
Avinash
Or it might send it back to sense and the whole process gets repeated.
Avinash
So, in a way, it is taking that decision autonomously whether I'm in a sense stage, when do I need to move, think, act, and then how the loop proceeds from there.
Avinash
But, all this is powered by the the true brain of the system, which is the LLM interaction layers, right?
Avinash
Because LLM enables the agentic AI system to perform these kind of actions, whether it's sense, think, or act, right?
Avinash
So, LLM typically understands your intent and then translate into specific actions, whether it whether you're in sense stage or think stage or act stage.
Avinash
And we have used LLM to to do the to to do that in a at a as an individual tool, but within agentic AI system, you can see how LLM helps here.
Avinash
And then last one is learning and memory, right?
Avinash
Because one of the true nature of an agentic AI system is to learn from what is going on, right?
Avinash
So, it has a learning capability, it has a memory capability.
Avinash
So, every interaction, every outcome makes the system smarter.
Avinash
So, past experiences led to I mean, lead to a better future decisions.
Avinash
So, this is not just uh, automating a task, right?
Avinash
You are creating a system which is evolving and can become better over a period of time.
Avinash
So, that's the true nature of an agentic AI system and I hope this architecture kind of explains what we are talking about.
Avinash
Now, with that, I think we have reached a stage where we can talk about what is how can we envision a agentic quality platform?
Avinash
And this architecture that I'm going to present in terms of quality platform has um, has come from our experience of building agentic AI platform within Pcloudy, right?
Avinash
So, in Pcloudy now, we have launched so many agents for different activities.
Avinash
Uh, so how we have reached that stage, how um, you can use this experience of ours to build maybe agentic quality platform within your enterprise ecosystem.
Avinash
And the way I envision is every organization at some point of time would need to have their own agentic quality platform.
Avinash
So, right now you have different tools, for example, you have an automation tool, you have your test management tools, you have your devops tools.
Avinash
But, moving forward, at some point of time, you have to think of your own agentic quality platform.
Avinash
It could be in-house, it could be maybe adopting a cot's tool, but you need to think about what would be my agentic quality platform.
Avinash
Uh, because that's the that's how the quality engineering world is moving towards.
Avinash
So, uh, without taking much time, let me just kind of reveal the architecture that we have thought about and uh, you can leverage it for your uh, organization as well.
Avinash
So, if you look at the again, it has multiple layers.
Avinash
Uh, the first layer is what we call it experience layer, right?
Avinash
So, experience layer defines how teams can interact with a platform.
Avinash
It could be natural language commands like I have a prompt interface where I can pass on the command that "Give me test cases for a particular feature, right?" Or say "Share me a report from my Jira." or "Create an automation script for my feature." So, I should be able to just provide my prompt and I should be able to get the outcome.
Avinash
And then you have You can have IDE plugins for some I mean, for developer integrations, you can have APIs to monitor your agentic AI systems.
Avinash
Uh, so, you can define your experience layer.
Avinash
Again, it's very I mean, it can be very enterprise specific, but there are various means through which you can have the experience layer, but natural language is going to be the primary experience layer for any organization.
Avinash
Now, you need to have the second layer, which we are calling it agent marketplace, right?
Avinash
This is your internal marketplace.
Avinash
So, you need to think of okay, for my quality engineering, what all different agents do I need?
Avinash
It could be public agents, it could be private agents.
Avinash
And public agent means, let's say, you might be using a selenium-based automation script creation bot, maybe from a third party.
Avinash
Like we also have a bot called Qpilot, so you can uh, integrate our Qpilot.
Avinash
So, that's a public agent.
Avinash
Uh, you can also build your own private agents based on your own data.
Avinash
So you have a repository of those agents at one single place which can be accessed by anyone through these interface or experience layer that I talked about.
Avinash
And these agents should be in a way where someone can install, fork, or customize, and then put it back in the agent marketplace, which is again your internal marketplace.
Avinash
But the third layer, which is the to being a true agentic platform is the multi-agent coordination.
Avinash
So one agent gives you a productivity, but as soon as you combine multiple agents, your productivity gains are at a different level, right?
Avinash
So for example, you can have a test creation agent which is helping you create your which is which is helping you automate your manual testing test creation process.
Avinash
But as soon as you I mean attach that agent to automation agent, then your gain is is is at a different level.
Avinash
You create your scripts, and at the same time you automate your script.
Avinash
Similarly, you can keep attaching agents, and then you create workflows.
Avinash
And that is where multi-agent coordination or orchestration platform become extremely important where you where you are able to attach multiple agents in a workflow and then execute it.
Avinash
So there are I mean industry tools like maybe LangFlow or N8N if you if you have heard or or if you have heard the announcement ChatGPT has launched its own agent orchestration platform.
Avinash
Or you can build your own.
Avinash
We also have agent orchestration platform to help you create workflows using the various agents.
Avinash
The fourth layer, which we also spoke about in the previous architecture diagram, is your AI fabric.
Avinash
So whether what kind of LLMs are allowed within your organization.
Avinash
It could be the standard public LLMs like OpenAI, Claude, or you can build your own models.
Avinash
Now, the the model training or model hosting is a is a challenging task, but it has become much easier in last couple of years from the time we have seen so many models getting launched.
Avinash
So it has become much easier for to fine-tune a model or host a model in in in even very compute efficient kind of systems.
Avinash
So you can think of creating our own custom models for specific use cases.
Avinash
And especially example, if you are a banking application banking organization, we have seen so many cases they are not allowed to use public LLMs.
Avinash
So you have to depend on some local LLM LLMs which you can host it within your organization.
Avinash
And you can train that for custom use cases.
Avinash
So so you you have your AI fabric, and you can provide access of different LLMs to to to the other layers that we talked about.
Avinash
And finally, this is all this is not complete until until you have your integration mesh.
Avinash
And by integration mean mesh we means ability to connect this agentic AI platform to various tools that you might already be using.
Avinash
So maybe your test management systems like Jira, or maybe a notification systems like Slack or Teams.
Avinash
Now with MCP as one standard protocol, those integrations also have become much easier.
Avinash
So so Jira already has its MCP.
Avinash
You can just quickly uh [Music] connect Jira MCP to your agentic platform, and those tools can be used within your workflow very easily.
Avinash
So you need to think about what are different tools that you are already using, and how do you create those integration uh plugins or integration MCPs to be able to use that within this whole agentic AI system.
Avinash
So with these five layers, you can think of creating your own agentic AI platform, and that at some point of time has to become one central tool which your teams can access.
Avinash
Right?
Avinash
And that is what I'm saying, it's intelligence everywhere and control nowhere.
Avinash
Uh we have seen organizations building test COEs.
Avinash
So test COE has test COE has to evolve from a traditional test COE that we have seen in past uh with different tools in silos to agentic AI platform where uh everything is bind together through all these layers that we talked we have seen here.
Avinash
Great.
Avinash
Uh now, this is not easy.
Avinash
The question is, it all looks great.
Avinash
How do we reach that stage?
Avinash
And like any other journey, you can't reach the top without following a specific path.
Avinash
Um so based on again our experience of developing it and working with so many organization, this this typically is a kind of maturity model that you can think about.
Avinash
So there are five levels here.
Avinash
Um uh you can first assess where do you stand.
Avinash
Are you at the foundation level?
Avinash
Are you at enhanced automation level?
Avinash
Most of the organization that we have seen are at somewhere between second and third.
Avinash
Third is assisted intelligence where organizations are already using some AI-powered test generation.
Avinash
They are using some pattern-specific use I mean they they have built some pattern-specific use cases to let's say detect bugs, or maybe find out impact analysis within their applications.
Avinash
So assisted intelligence is where a lot of organizations exist today, but many are still at level two.
Avinash
Now from level two, you can't reach level five immediately, right?
Avinash
So you need to think about okay, from level two, how do I reach level three.
Avinash
If you are already at level three, how do you reach level four, which is the orchestration part of it.
Avinash
And then from four, how do you how do you reach five, which is the self-evolving where it's more about learning from what's going on and learning from all the data that you are collecting on a continuous basis.
Avinash
So you need to think about where do you stand, and then plan accordingly.
Avinash
Typically, I mean at a very very high level if I have to predict, it can take anything between 6 months to 24 months depending on the pace that you can adopt within organization.
Avinash
Again, it it varies.
Avinash
Uh some organizations are very fast.
Avinash
Some organizations have a lot of approval hierarchies.
Avinash
So it can take between 6 months to 24 months.
Avinash
But the point is you need to assess where do you stand today, and then start taking steps to move to the ladder that we see here.
Avinash
So I hope this gives you some idea of where how do how should we plan this journey.
Avinash
Like I said, you can't reach level five.
Avinash
We haven't reached level five.
Avinash
are at level four, and we hope to reach level five in some time.
Avinash
We are working towards it.
Avinash
So plan your journey accordingly, and start small, start doing experiments, but you need to start the journey sooner than later.
Avinash
Now, having talked about all that, uh this is more like painting a picture, right?
Avinash
This might sound like too good to be true, but we are we are reaching this stage.
Avinash
So this is just imagine maybe next 2 years, can you have this kind of workflow?
Avinash
And we have already seen a lot of organizations.
Avinash
For example, if you look at Facebook case studies, they have already achieved a stage where a lot of it that you see here is is already happening.
Avinash
So this is kind of code push to testing kind of workflow.
Avinash
And each block that you see here is a kind of agent, right?
Avinash
Not kind of agent, these are agents.
Avinash
So let's say there is a code push that happens for a particular feature, um which is around PCI compliance for a banking application, right?
Avinash
So it it the code analysis engine kicks in.
Avinash
Code analysis agent kicks in.
Avinash
And then it analyzes uh what are the impacts, and it then sends to the impact analysis agent.
Avinash
And it finds out okay, there are so many services which are impacted.
Avinash
Now based on that, it sends to a test generation agent.
Avinash
At the same time, environment setup agent.
Avinash
These two are happening in parallel.
Avinash
And as soon as these two agents have their work done, the execution starts.
Avinash
And based on if it passes, it's fine.
Avinash
If it fails, then it goes into a triage goes to a triage agent where it figure figures out whether fix is needed, or we need to escalate it.
Avinash
Again, these are it could be one agent, or it could be a separate agent.
Avinash
That depends on the architecture.
Avinash
And then a overnight regression agent kicks in.
Avinash
And finally, your release happens.
Avinash
So this is a kind of workflow that you can envision.
Avinash
And we we are we are seeing this possibility in in near future.
Avinash
Like I said, many organizations have already adopted this and have been able to achieve it.
Avinash
So it also depends on the maturity of how good your processes are.
Avinash
But if you have those processes, you can start thinking of those agents and think of this as an orchestration.
Avinash
Like I said in in my previous slide also, the real productivity gain happen happens when you have these multiple agents working together in form of workflow.
Avinash
So, think of Can you paint this picture within your organization?
Avinash
So, the question is again going back to exponential impact.
Avinash
Like I I kept harping on this point that 1 + 1 in the world of AI could be two, it could be three, it could be 11.
Avinash
It's your choice, right?
Avinash
So, agents don't just add individual value.
Avinash
As soon as you attach multiple agents, they multiply and the effectiveness of the overall ecosystem is is can can become exponential.
Avinash
And that's the true nature of a AI-driven system, which was not possible with the previous technology shifts that we had seen.
Avinash
So, we can we can think of exponential impact with the agent true agent equality platforms.
Avinash
And if you're able to build build it the way we envisioned in the previous slides.
Avinash
Now, would like to quickly kind of talk about the elephant in the room, the new QE professional in this changed space, right?
Avinash
And I talked about this a little bit in the beginning as well that the role is changing, right?
Avinash
And another insight that we have seen is in spite of all the fears that the the with the AI can QEs be replaced, I mean I'm finding it more and more difficult to believe that replacement is possible.
Avinash
But at the same time, elevation is definitely needed, right?
Avinash
And elevation in terms of moving from writing test scripts to becoming an intelligent curator, right?
Avinash
From maintaining automation frameworks to becoming quality strategists, from doing typical your I've been debugging of automation scripts to becoming agent orchestrators.
Avinash
So, there are different expectations that will come into play as we move towards this whole agentic driven quality engineering organization.
Avinash
So, from test engineer to quality strategist to AI curator or AI trainer is the kind of path we'll see more often which will be expected from all of us.
Avinash
And with the systems that are in that are available to us, with the knowledge which is available to us, I think we all have an opportunity to scale and elevate ourselves.
Avinash
And I'll again go back and repeat that statement that this is truly a very unique time in the history of the technology where we all can become creators.
Avinash
We don't necessarily need to be stuck to it being executors.
Avinash
You can become builder, you can become creators.
Avinash
That's the power which is provided to us with this AI wave.
Avinash
So, what's the choice do we have?
Avinash
You can still wait and see or look at other success stories to emerge.
Avinash
Or the option B is that you start today, right?
Avinash
You lead the change.
Avinash
And this can happen through experiments.
Avinash
This can happen through smaller steps.
Avinash
And that is how we also achieved this whole agentic AI quality platform in last two years.
Avinash
We went through a lot of experiments.
Avinash
We made a lot of mistakes, but we kept trying new things and finally I'm I'm not saying that we have achieved a perfect stage, but we have stage where we can say we have a true agentic quality platform with us.
Avinash
So, it's important that you start trying, experimenting with smaller agents, use cases, build smaller workflows, and then see the outcome of it and then keep iterating.
Avinash
That's the key to success.
Avinash
There's no other shortcuts that I can suggest based on my experience.
Avinash
And again a quote here that there was a the best time to start was maybe yesterday.
Avinash
The second best time is now.
Avinash
So, with that thought, I would like to conclude the discussion or the session that I wanted to talk about.
Avinash
We'll be happy to understand your perspective and if you have some different thoughts, if you've tried something new within your organization, feel free to share.
Avinash
We can we can take up questions if you have any.
Avinash
Would also love to interact later, so you can connect with me on LinkedIn or share your you can share share and shoot up a mail to my email ID.
Avinash
It's avinash.tiwari@pcloudy.com.
Avinash
Happy to happy to have you here and talk about one of the topic that I'm so passionate about.
Avinash
Thank you so much.
Avinash
Over to you, Dinakar.
Dinakar
Great.
Dinakar
Great.
Dinakar
Thank you, Avinash.
Dinakar
Thank you for that insightful session.
Dinakar
You've covered a lot of aspects in terms of building agentic platforms for our quality systems as well.
Dinakar
So, we'll just get in quickly into a small a short Q&A section.
Dinakar
I know we're running out of time, so we'll make it quick.
Dinakar
I have I do have a couple of questions from the audience.
Dinakar
So, let me just shoot out one to you.
Dinakar
Here's the question.
Dinakar
Can you give an overview of Pcloudy's capabilities as an agentic quality platform?
Avinash
Right.
Avinash
So, again, if you remember the architecture that I shared with you, we have built multiple agents across different use cases that we serve.
Avinash
Of course, we are into the space of digital testing.
Avinash
So, our agentic quality quality platform is focused in that area.
Avinash
So, within that, for example, we have a test generation agent.
Avinash
From that, you can create a manual test cases.
Avinash
We have a test automation agent, which we call it Qpilot.
Avinash
So, from that, you can generate your automation scripts.
Avinash
We have a performance analysis agent.
Avinash
So, once you install your application, it can track your performance matrices and then we have analysis agent on top of those that performance agent, which can analyze and gives you recommendations to improve your performance.
Avinash
We also have visual AI agents.
Avinash
So, depending on the the use cases that you have within the digital testing, we have various agents in place.
Avinash
But one of the interesting aspects that we have built and we are going to release it very soon is this orchestration platform, right?
Avinash
So, as a QA, how quickly can you build your agents for the QA use cases?
Avinash
So, for example, you might want to create the workflow that we just saw in the presentation, right?
Avinash
Starting from, let's say, impact analysis to creation of test cases and then based on that, find your right automation test case and execute it.
Avinash
That's the workflow that you might like like to create.
Avinash
How how can you create it easily?
Avinash
That's the orchestration platform we are we have built and we're going to launch it very soon.
Avinash
So, we have a full stack of agent agentic AI platform, various agents for use cases, and then you can combine those agents to create workflows.
Avinash
We another thing I would like to highlight is we are one of the We are in a unique place here because we support both public LLMs.
Avinash
So, let's say you can use OpenAI or some of the other popular public LLMs like Claude.
Avinash
But we also have the ability to host a local LLM for your various use cases.
Avinash
So, if you're an enterprise and you are not supposed to use any public LLM, we can host our solution based on a local LLM and that will be like dedicated for your tasks.
Avinash
So, in that sense, we have a true agentic platform starting from the LLM fabric to to various agents and then orchestration.
Dinakar
Brilliant.
Dinakar
Brilliant.
Dinakar
This is another question I think around frameworks and how do we do this internally.
Dinakar
So, if you if we wanted to create our own domain-specific agent, for example, a banking app compliance agent, what skill set and frameworks would we need internally?
Avinash
Yeah.
Avinash
Yeah, I mean building custom models and agents on top of it is slightly involved task, but it's becoming easier and easier as we as we are seeing the evolution of different tools and technologies in this space, right?
Avinash
So, there are various open-source models that exist.
Avinash
And first, you can maybe download that model.
Avinash
And there are various platforms that are available where you can quickly download like LLM Studio, etc.
Avinash
And then start doing some experimentation as to how good those open-source models are for your specific your cases.
Avinash
First, you can do that evaluation and after that you can think of okay, if it if it's not giving you the right accuracy, you might have to add certain data and do some fine-tuning.
Avinash
Now, fine-tuning requires a specific expertise, so you can take help of your AI team or LLM engineers if you have.
Avinash
Or there are I mean enough material available if you want to get into detail and do it yourself.
Avinash
Uh but that's slightly involved task where you can do some fine-tuning on the data that you have.
Avinash
Some cases you don't have to do fine-tuning.
Avinash
You can you can use the rag pipeline and add your data to the existing open-source LLM and it might help your use cases.
Avinash
And then on top of it you can use some agent building platform like maybe LangFlow or QCrew or you can write your own scripts to create your agents.
Avinash
So, it's not so difficult at this moment of time.
Avinash
There are various frameworks and tools already available to make it as cordless as possible for you, but you need to understand what you're trying to do and you should be very clear about the steps that you need to take.
Avinash
Like I said, first evaluate an open-source LLM and then whether you want to use rag or do some fine-tuning.
Avinash
Those strategies require a little bit of understanding, but now with the help of ChatGPT you can also figure out the strategies and then start following the steps.
Avinash
Um so, that's Again, going back to that point that we all can become the build builder that we all wanted to AI has given us that flexibility.
Avinash
So,
Dinakar
very interesting point here then occur because this is where I see a lot of work happening in the AI space, building custom models, building use case specific models or agents on top of it is one big ask which will come our way from within every organization.
Dinakar
And I think we we all need to be ready for it.
Dinakar
Another interesting question.
Dinakar
I mean lots of questions coming up.
Dinakar
So, this one says, you know, being QA engineers, how can we test LLMs currently as you know, as most of the organizations are still in the developing or the experimenting phase.
Dinakar
So, building AI agents means we are involved in development as well.
Dinakar
Is that right?
Dinakar
Is is the question?
Avinash
That's a that's a that's a new emerging area and an opportunity for all of us as as quality professional to learn the ability to test AI agents itself.
Avinash
Right?
Avinash
So, for example, everyone is building AI agent.
Avinash
AI agent ultimately behind the scene either is interacting with LLM and then there is some code.
Avinash
Now, that code can go wrong, right?
Avinash
How do you make sure that the AI agent is performing the way you envision it to be or it's performing the task which what which it was supposed to be.
Avinash
Plus with AI systems there are other new parameters.
Avinash
For example, the hallucination, testing for hallucination, testing for bias, testing for security of the data.
Avinash
All that are new emerging areas in terms of the the testing parameters that you might have to look at for AI agents.
Avinash
So, um and and that to be very very frank, to to call it testing is not the right word, right?
Avinash
I I I mean we do this all the time.
Avinash
We call it evaluation.
Avinash
So, we are moving from testing to evaluation because you can't say for AI agent or for a model that it is incorrect or correct.
Avinash
We are moving from that to you you have to say it's 80% meeting the requirement.
Avinash
It's 90% […]
Avinash
You have to talk in terms of accuracy level.
Avinash
You have to talk in terms of hallucination levels.
Avinash
So, it's a completely different paradigm, but I think with the with the evolution of every task being done through AI agents, testing of AI agent itself will become a new area which we all need to figure out how to do it.
Avinash
At Pcloudy we have developed a agent evaluation platform as well.
Avinash
If you'd like to test your AI agents or your custom models, you can use our platform to do that as well.
Dinakar
I know we're running short of question I mean I we're running short of time, but I think just this last question maybe Avinash if you can, you know, talk about how as AI agents take on more execution and decision making, how should the QE engineers evolve their skills and you know, how do they stay relevant and lead in this whole new paradigm?
Avinash
Like I said, I I touched upon that point in one of the slide as to how the role would elevate for all of us.
Avinash
So, from writing scripts to agent orchestrators to agent testers, there are various new expectations that we all would have and I think we are already seeing that.
Avinash
Um [Music] So, we all need to learn how to use AI effectively.
Avinash
That's one of the basic point I would like to address.
Avinash
Um learn a little bit of maybe um maybe a little bit of coding as well to be able to effectively use AI systems, etc.
Avinash
So, some of these expectations are emerging in terms of what is how the roles will evolve for quality engineers as well.
Avinash
But in a nutshell, yeah, I mean um the roles are changing for sure.
Avinash
Um but I definitely feel that we are not in a stage where um there is a risk of replacement.
Avinash
There is a risk risk of replacement if we don't upgrade ourselves.
Avinash
That's that's definitely true.
Avinash
But if we upgrade ourselves, it's opening up a huge new era for us and like I said, it's an opportunity for for all of us to become creators and builders.
Avinash
Absolutely.
Dinakar
Well, it's an absolute pleasure pleasure hearing from you, Avinash, and you know, thank you for sharing all these valuable nuggets on building agentic systems for QE.
Dinakar
And your presentation has covered a lot of practical insights.
Dinakar
And I think the goal of this whole presentation, this whole webinar is to get to start thinking and start to have these discussions back home in our in our work desks or in our meetings in our development product meetings.
Dinakar
I think that's the goal for everybody to take back here.
Dinakar
And I'm sure while Avinash was speaking you would have had that eureka moment or chalked out a few ideas to work on.
Dinakar
And that's the goal to get you thinking and gather your teams to start building on these ideas and have discussions around it.
Dinakar
And so, feel free feel free to reach out to us.
Dinakar
You've got Avinash's email and you've got you can you can reach out to us at info@pcloudy or directly reach out to Avinash and we'll get back to you at the earliest.
Dinakar
And so, yeah, thank you, Avinash.
Dinakar
Thank you for your time.
Dinakar
And also thank you to each one of our participants who has been a very engaging audience asking these questions, thought-provoking questions.
Dinakar
So, thank you everybody for sparing this time in this afternoon and being an engaging audience.
Dinakar
And happy building.
Dinakar
Thank you so much.
Dinakar
Have a good day!
