Dinakar
Alrighty.
Dinakar
Uh hello everyone and welcome to Pcloudy's webinar on building a continuous evaluation framework for testing AI agents.
Dinakar
My name is Dinakar and I'll be the host for today.
Dinakar
Before we get started, I would like to go over a few housekeeping rules.
Dinakar
You may have noticed that your line is currently muted and if you have any questions, you can submit your questions during the webinar using the Q&A option available.
Dinakar
We will answer your questions as part of the Q&A sections toward the end of the webinar.
Dinakar
Also, please note that this webinar is being recorded and we will send the link of the recording to share with your colleagues or watch again.
Dinakar
With that said, let me introduce the speakers for today.
Dinakar
Uh we have Avinash Tiwari.
Dinakar
Avinash is a renowned thought leader, a recognized keynote speaker and the co-founder of Pcloudy.
Dinakar
He has 20 plus years of experience in product development and testing.
Dinakar
He brings with him a passion for emerging technologies and quick adoption.
Dinakar
Both of which have solidified his reputation as a leader in the testing space and the agentic AI as well.
Dinakar
We also have Jessica today who will be talking us through the demonstration of the agent evaluation platform.
Dinakar
Uh and Jessica is a test evangelist at Pcloudy.
Dinakar
She has four plus years of experience in the IT industry and prior to her role at Pcloudy, she has worked with LTI Minry as a pre-sales consultant and has a keen expertise in data analytics and various automation tools.
Dinakar
So without further ado, let me hand it to Avinash to kick off this exciting webinar.
Dinakar
Over to you Avinash.
Avinash
Uh thanks Dinakar for hosting this webinar and thanks for the introduction.
Avinash
Um hello everyone.
Avinash
Uh I am really excited to be here and up front I want to apologize for postponing the webinar last time.
Avinash
It was supposed to happen two weeks back but because of some personal uh emergency I had to postpone this.
Avinash
This topic is very close to my heart and I wanted to do it myself.
Avinash
if I wanted to present to all of you and hear your views and feedbacks.
Avinash
So I I thought let me do it myself and here we are.
Avinash
Uh having said that uh we are living through an living through one of the most transformative times in the context of […]
Avinash
Uh the amount of change that we are seeing around us is just insane to say the least.
Avinash
And to quickly get a sense of the pace of change, why don't we do a quick thought exercise?
Avinash
Maybe it's IQ test for you.
Avinash
[laughter] No, I'm just kidding.
Avinash
Just close your eyes and I want all of you to think every AI news, announcements, uh tools that you have come across in last 30 days.
Avinash
Yeah.
Avinash
Just last 30 days, not last one year.
Avinash
I I want you to focus on last 30 days.
Avinash
So just for 10 seconds, think about it.
Avinash
Maybe note it down for yourself and see how many can you count.
Avinash
Okay, I think you can open your eyes or maybe just think about the list that uh you saw.
Avinash
U to give you a sense of uh like I said the pace of change has been so insane.
Avinash
Um this is one of the picss that was circulating though it's a it's a dummy pick.
Avinash
It's it you can call it a AI slop but in a way it represents that even the tech titans had to find some time off from the uh change which is happening around all of us and maybe discuss how do we slow it down.
Avinash
So um but yeah I mean jokes apart I think the change has been going through a rapid pace and I think we all are seeing the impact around us.
Avinash
Um so something like in last 30 days we saw Gemini 3 release we heard so much about nano banana pro openai released agent kit there's a lot of debate around GPU TPU and you would have heard numerous tools releases in the space of AI testing or otherwise uh the point is that this change is real right and it's impacting all of us u in any industry In any function that we see around uh there are agents everywhere.
Avinash
You talk about coding, you talk about learning, you talk about sales task, marketing task, you pick up any industry, uh people are experimenting with different agents where they can perform some of their uh use cases.
Avinash
Um there are tools coming up with to help you uh implement those agents.
Avinash
So the point is that this is not just news anymore.
Avinash
This change is real, right?
Avinash
And we are already seeing the impact across functions, across industries.
Avinash
There is so much talk around what's going to happen with the jobs that we have and all sorts of uh news I'm sure you all are aware of.
Avinash
Having said that, uh I I quickly wanted to touch upon or brush upon some of the terminologies because we'll be using these terminologies in in our uh discussion today and it's important that we understand u the proper context of it because these terminologies are also used interchangeably.
Avinash
Um so there are couple of concepts that I wanted to clarify here.
Avinash
First AI agents, right?
Avinash
We all know uh what AI agents are.
Avinash
And then there's another term called agentic AI.
Avinash
And there's third term called workflow, agent workflows or agent orchestration.
Avinash
I mean again people use it differently based on the context that they are in.
Avinash
So if we talk about AI agents, I think we all are very I mean we all are very familiar with chat bots.
Avinash
We have use open AI or maybe Gemini.
Avinash
And uh when we ask these chat blocks to perform a specific task, they are able to do that, right?
Avinash
And that's a that's typically definition of an agent which is very task based.
Avinash
It waits for a command.
Avinash
It generally works alone, right?
Avinash
It's isolated activity where you ask for a specific command and it responds based on what you have asked for.
Avinash
it it has its own intelligence but typically it follows certain rules while when we move talk about agentic AI it's more proactive right it's goal oriented one of the major difference between agent and agentic is goal oriented and very in a very simple example for example you can say in case of an agent that okay generate a test case for me right for for login functionality or maybe my checkout functionality it will generate the test cases for you to agent I say that go and test my uh checkout functionality.
Avinash
So it's very goal oriented.
Avinash
So in this case it can also think of okay what are the test cases that I need to generate and when I generate those test cases how do I go and execute it.
Avinash
So it focuses on goal rather than a specific command right.
Avinash
It's also collaborative in the sense that it can work with multiple agents.
Avinash
So like I said in my previous example first it can generate the test case and then go and execute it.
Avinash
So internally it could be two different agents coordinating together.
Avinash
and it also has its own learning capability based on the memory and and context that uh it learns from.
Avinash
So there is a fundamental difference between agent and agentic AI.
Avinash
And I think in the context of what we are talking today both are important.
Avinash
The the difference between two are extremely important because when you build a framework you need to understand you're testing an agent or you're testing agent TKI platform.
Avinash
And the third concept which I've not covered in the slide here is the agent […]
Avinash
So you have heard I mean you would have heard about tools like Netin or even OpenAI has release operator etc where you can create agentic workflows right where you combine multiple agents to perform a much bigger task um and and these agents coordinate with each other pass data from one agent to other and finally perform a particular task.
Avinash
So that's called agent orchestration.
Avinash
Now just to understand a little bit on the agentic AI I think uh it's important that we understand the architecture of it has orchestration layer like I said in agentic AI multiple agents can coordinate coordinate with each other so orchestration layer takes care of which agents to invoke now when it when it try when it tries to decide it typically follows three-step approach sense think and act the sense means what's happening think what should I do and act make it happen and these three are not in sequence I mean it can start with a sequence but it can just uh move between these stages depending on the um task and how the decision is made so for example after act it might realize that okay I don't have enough information I need to go back to sense again or I need to go back to think again so this is an iterative process but at end of it After these phases are over, your goal is […]
Avinash
So this since think and act sequence is happening with the help of LM interaction layer.
Avinash
So it could be your public LLMs like OpenAI or Gemini.
Avinash
It could be your internal custom enterprise LLMs as well.
Avinash
And then there is a learning and memory which is which maintains the context of what's going on and it can also help agent learn for future tasks.
Avinash
So that's typically the agent AI architecture.
Avinash
Again, this is important to understand because when you are building evaluation driven how do we uh create the workflows and how do we make sure that uh there's there are agent hands off and all that.
Avinash
We we'll cover those aspects as we move along.
Avinash
Um so some understanding of this architecture is helpful in in when you are building those frameworks.
Avinash
with with these concepts uh in place uh I think we are all set to discuss how what and why of agent testing and uh I'm I'm right now calling this term agent testing but u I'm not uh pro I mean propagating this term as agent testing itself we'll try to evolve from agent testing to a different terminology and there's a reason why we are starting with agent testing and we'll navigate to some other terminology.
Avinash
So what's what's different, right?
Avinash
I I I have seen so many cases and we have built so many agents and when we started testing it, we also faced this problem, right?
Avinash
Um that uh my QA team tested an agent and they said okay I have executed my test cases for the agent is ready for production go ahead and deploy it.
Avinash
After the deployment happens, uh we see that there are issues.
Avinash
Uh customers have started reporting problems.
Avinash
It's giving inconsistent information, wrong information and yeah all hell loose break right.
Avinash
Um so that's a scenario which we have seen very often in the world of agents.
Avinash
Um so the question is if all tests passed what went wrong?
Avinash
And that's what the fundamental difference comes between traditional software testing and AI agent testing per se.
Avinash
Uh in traditional software we know right it's very deterministic.
Avinash
So for example 2 + 2 always will be four and if you do a run and your software has given 2 plus 2 you're pretty confident that next time also it will perform the same task.
Avinash
I mean there are some situation where it might not but more or less after a run you are confident that my software is working fine.
Avinash
I have tested across these test cases.
Avinash
I have given these inputs and it's giving me uh those outputs.
Avinash
I'm confident about it.
Avinash
So results are predictable.
Avinash
Uh when we talk about any traditional software and that's the world we have been living in uh for quite some time.
Avinash
Um cut to AI agents and the current world that we are in.
Avinash
U the biggest difference is that the results are nondeterministic and we know that LLMs itself are nondeterministic.
Avinash
So for example when you say or the test case says that okay test for a prompt what's what's my refund policy for the first run it can say okay we we offer 30 days refund in second run it can start with a completely different um uh statement a third run again it can be a different outcome and each time you run the result will be slightly different right um how do you test it right how do you say that which result is good which result is bad and should I just rely on one run or should I look at multiple runs and then figure out the accuracy of it.
Avinash
So these are the questions that we are dealing with when it comes to testing of AI agents.
Avinash
So very clearly in a in a in a nutshell we are dealing with a world which is nondeterministic.
Avinash
So you can't use a pass or fail criteria on nondeterministic system.
Avinash
And I I went through this mindset shift for I mean it took some time for our teams to have this mindset shift when they were testing the agents that we had built for PC loading that initially it was mostly about pass or fail and then they realized that no it's pass fail is not enough we have to move to something else and that's the shift every QA team has to go through when they think about testing of agents or building evaluationdriven uh testing frameworks.
Avinash
So what are we talking about?
Avinash
I mean again this is not fully comprehensive but there are certain important aspects which makes the uh agents or LLM's uh um for example hallucination.
Avinash
Uh we know that agents can make up answers.
Avinash
So if someone has asked a question and it doesn't have an answer it will it might not say that I don't know.
Avinash
it might just make up the answer and and completely in a very confident way and c customers or the end user might think that this is the answer which is right for me right so hallucinations are a big problem with models and that impacts the agents outcome as well inconsistency I mean like we discussed in the previous example every run can have a completely different answer we have seen you you ask the same question to openAI and you run it multiple times the answer you will see a different answer Right?
Avinash
So there is inconsistency in the u outcome of the same prompt if you run it multiple times.
Avinash
Uh the third problem that we see is the context length.
Avinash
So for example, if you're interacting with an agent and it's a multi-turn conversation that you're doing is your um designed to handle a very large context length.
Avinash
We know for example I mean right now Open Penni supports milliondoll token as the context length which has which is more than enough but if you're dealing with let's say a smaller custom LLMs typically these are 24k tokens or in some cases 50k tokens now if you're if you're doing a multi-turn conversation and especially let's say you're doing some code generation one output itself can consume so much of your context length so uh the agent after a period of time can can lose the previous context and start answering wrong right so are you maintaining the context in the testing uh and how much can it handle right it's like load testing that you do for applications that how much can it handle in terms of load so context though it's not load but after a period of time you have to see whether it's able to handle the uh context and it's able to remember the memory that you have given to the agent uh the last one is agent handoff failure I mean this is more in the context of agenti system and that's the reason I talked about the concept where there could be multiple agents working together.
Avinash
So how is maybe the agent in itself is working fine but what if one agent has to pass the data to other agent is the handoff working correctly right uh and it could be uh handoff of the context it could be handoff of the outcome is the handoff working fine right so two agents independently can work together but as soon as you combine there is an issue with the handoff and the overall outcome is not correct so u u these are the I mean these are certain areas which makes the agent testing very different from our traditional testing.
Avinash
Um and like I said pass fail is not the criteria which um which truly defines the testing of the agents.
Avinash
So so then what are we talking about?
Avinash
I mean if we if you have to summarize the difference between uh traditional testing and evaluationdriven development or evaluation driven testing what's the difference that we talking about?
Avinash
So in traditional testing it's all about does it work while in evaluation driven it's how well does it work right in traditional testing it's binary pass or fail in in in evaluation driven is continuous you have to rate it over a uh whatever grade that you want to give it could be 0 to 10 it could be 0 to 100 so you have to grade okay my outcome is 80% accurate and over a period of time it's increasing from 80% to 85% to 90% % so you have to continuously test and then look at uh the pattern and see whether it's improving.
Avinash
So it cannot be pass or fail.
Avinash
Traditional testing again fixed expected output uh in case of evaluation driven it's whether it's acceptable quality threshold.
Avinash
So when you build a framework you have to define what is the threshold which is acceptable to me.
Avinash
It cannot be 100%.
Avinash
We have seen uh in the current world 100% is not the case.
Avinash
So you have to decide what is the acceptable quality threshold.
Avinash
What is the variance that you are seeing?
Avinash
In traditional testing, you test once you get the output.
Avinash
You're pretty sure that there then the software will perform the same way if you test it next time.
Avinash
But in case of evaluation driven, it can vary.
Avinash
So what is the variance?
Avinash
You run it for multiple times and see whether there's a variance and the variance is acceptable or not.
Avinash
Um here it's all about in in traditional testing again it's all about functional validation and evaluation driven behavior.
Avinash
We discussed that uh in in traditional testing it's all about deployment checklist that okay test passed I mean sanity passed regression passed all that uh checklist is done and then you're ready for deployment in evaluation driven it's all about quality benchmark gates right so how much is your variance is it acceptable how much is your accuracy threshold met or not and all that criteria is measured and after that you decide whether to push it or not.
Avinash
And to give you another I mean sense of why AI agent testing becomes extremely challenging is I'll just give you a small example even if you have tested it uh there is a new version of the model it can completely change the outcome of it.
Avinash
Um and to understand that maybe you can go and look at something like model temperature.
Avinash
There's a uh every model has its temperature set.
Avinash
So when you are calling an agent using APIs, you set the temperature as well for the model.
Avinash
If you set the temperature low, it's more deterministic.
Avinash
If you set the temperature high, it's more random.
Avinash
So slight change in the temperature can change the outcome of your agents.
Avinash
So sometimes the model new version is released and they have changed their system prompts.
Avinash
Maybe their temperature settings are slightly different and it can change the outcome of your uh agents.
Avinash
So that's the challenge we are seeing with the testing and evaluation of the agents here.
Avinash
So what's the framework that we are looking at right?
Avinash
What's the evaluation framework?
Avinash
I mean again this is a broad uh area that we are covering and based on our experience we have listed certain uh areas here.
Avinash
So first is accuracy and we talked about accuracy in terms of the outcome quality of the uh agent response.
Avinash
uh and it has to be graded right I mean it could it can be on 0 to 100 it can be 0 to 10 whatever parameter that you choose to uh rate your accuracy on consistency so I'm sorry consistency means if you run it multiple times how consistent the result how what's the variation uh the third piece is reasoning quality right nowadays you have seen one is the response of the agent and then second piece is what what is the reasoning um quality that agent agent has used to generate the outcome right so you want to validate both has it used right reasoning to generate the outcome so even if let's say outcome is u I mean if the outcome is not right you had you have to look at what was the reasoning used and why the reasoning was not correct so that's another new dimension to the agent testing which we see in today's um uh in today's context the fourth is context retention we talked about if you're doing a multi-turn conversation you have to ensure that agent is able to retain the context uh even in cases where there is a agent AI system where you're passing data from one agent to other agent the context retention is retention is extremely important otherwise the next agent can start behaving uh uh without using the previous information right and it can generate a different outcome and the last part which becomes very important in case of enterprise systems is safety and reliability, right?
Avinash
So, for example, ination, your bias, your boundary adherence, etc.
Avinash
are extremely important criterias uh when you're really releasing an agent u for your customers.
Avinash
Uh so these are uh certain broad uh areas that we need to consider which uh are extremely important uh when you are building a u AI AI agent evaluation framework.
Avinash
So what's the thought process when you think of uh building a framework for it's a testing of your systems AI systems or agents or LLMs whatever I mean for depending on your enterprise need the first part is definitely setting the benchmark right so like I we we discussed multiple times that what's the quality benchmark that you're looking at what's the acceptable accuracy for you what's the uh and again the every Organization can have different criterias.
Avinash
Some organizations maybe just can focus on accuracy.
Avinash
But if you're talking about enterprise, they will focus on various other parameters like empathy score to your bias score and and all sorts of parameters.
Avinash
So what are the benchmarks that you're looking at right and again depends on the use case.
Avinash
There are certain use cases which are very complicated.
Avinash
We know that the AI systems are not so good.
Avinash
So maybe the initial acceptance criteria could be 70%.
Avinash
uh and then it improves over a period of time if you do more training to the uh models etc.
Avinash
So depending on what stage you are in you need to sit with your maybe project management teams or your uh technical teams to understand what's the metrics we should be uh tracking for uh my evaluation framework.
Avinash
Then it's more about your actual evaluation where you need to do run you need to run comprehensive tests.
Avinash
So there for example you need to create your test set what are the different kind of prompts I'm going to try what is the expected response which is which is meeting the qual quality metrics that I have set for myself right uh what are the dimensions that I need to consider all that has to be part of your uh uh evaluation data that you will be building right and third is the continuous monitoring right continuous testing and monitoring uh Like I said uh in case of traditional software let's say if you have tested and until the application goes through any change you are more or less confident that the application will perform uh based on the testing that we have done.
Avinash
But in case of models very small changes which are not even uh where our agent code uh even if the agent code is stable some small change in the model parameters or temperature or system prompts can completely break it.
Avinash
So it's very important that there is a continuous monitoring and evaluation in place for the uh for the framework.
Avinash
U with that u just a quick summary of agentto agent testing right and this is another area that is becoming a challenge is okay I can test one agent but then how do we test a workflow right where agent to agent testing handoff uh handoff testing apart from the other checklist that we discussed is extremely important right so for example if customer query it first starts with a sales agent then it goes to a technical agent then third it calls a billing agent how do you ensure that that workflow is tested right you can ask a prompt and then get the response but how do you ensure that uh each agent is working properly to provide the right response if the if you've got the right response okay you are confident that maybe the system is working fine but if the response is not good how do you know that which agent has failed Right?
Avinash
So you need to build a system where you are getting uh uh results from all different agents and you are able to trace and track uh and then report those issues uh in a more correct way.
Avinash
Right?
Avinash
So that's another part of uh the evaluation that we are looking at now.
Avinash
U we are we are kind of ready for an action.
Avinash
So we have discussed various aspects of agents.
Avinash
uh the testing needs and uh different dimensions of testing that we need various challenges that uh can come across.
Avinash
So we have uh built a agent not testing platform.
Avinash
I would now now call it or change the term to agent evaluation uh platform and now you can relate why I'm calling it an evaluation because it's not testing right.
Avinash
Um however these are semantics and we can discuss and debate.
Avinash
Point is that uh the the approach is different.
Avinash
The um the way we have to look at the outcome of testing is very different.
Avinash
So we have decided to call it a agent evaluation platform.
Avinash
Um where we test agents across different parameters.
Avinash
Uh of course these parameters will evolve as we grow and and depending on the customer feedbacks etc.
Avinash
But it's quite comprehensive in terms of the uh areas that we have considered.
Avinash
So why don't we just let's see it in action.
Avinash
So I'll pass it on to my colleague Jessica.
Avinash
Jessica over to you.
Jessica
Um let's let's see what we have in store with respect to Pcloudy agent evaluation platform.
Jessica
Thank you.
Jessica
Thanks Avinash.
Jessica
Um so yeah uh if you could please stop sharing so that I can Yeah.
Jessica
Thank you.
Jessica
Yes.
Jessica
So uh now let's get into the demo.
Jessica
Uh we'll be evaluating Pcloudy's own AI assistant uh which is called Alfie.
Jessica
So some of you might have already uh used it if you're users of Pcloudy already.
Jessica
Basically what this AI agent would do for you is that it would help you it's a it's an AI agent that would assist you in your automation as well as accessing devices.
Jessica
So be it say you want to list out what are the available Samsung devices or what are the available uh iOS devices that I I can connect to or say you want to generate some capabilities um in order to execute your automation scripts you can do that or sample automation scripts for you to run your automation.
Jessica
So basically just to give you an outline of what Alfie is that is uh Pcloudy's uh AI agent to assist you in connecting to devices and generating capabilities and scripts right so let's just I'll just run maybe a sample uh prompt for you to see how it works so here we are generating an APM script to test the signup uh feature for an application right for Facebook and you can see that it has generated a sample script with the capabilities for me to connect to and run on Pcloudy and it's also generated the script for me and it also highlights where I need to substitute uh the device name and other details for me to run on my platform on uh from your framework right similarly say I want to generate device capabilities.
Jessica
So capabilities is nothing but a code snippet that would help me uh connect your say your APM scripts uh or your existing framework script on the Pcloudy platform.
Jessica
So again we're generating capabilities for an iOS […]
Jessica
So we've given the prompt and again it has generated the capability code.
Jessica
You can copy this and use it in your code.
Jessica
Again, say you want to just see what are the list of Samsung devices that are currently uh available for me to connect to, right?
Jessica
Uh so I'm giving that prompt now and it will go ahead and give me the response.
Jessica
Okay, so Samsung Galaxy S22 is available uh in the in this particular instance and I can directly connect to it from here.
Jessica
So this is how our uh Alfie agent works.
Jessica
Now as was elaborated by Avinash right we cannot do a traditional pass or test um a pass or fail kind of testing here.
Jessica
So how would we go about evaluating it and that's where our Pcloudy's a AI agent evaluation would come in right so before we begin our evaluation what are the data points what are the things that I would be needing uh right so firstly is the evaluation framework uh that Avinash mentioned on the consistency context hallucination and so on which we will get into and next we would need an expected response data which is basically We are giving a set of prompts and what would be the expected response right in CSV format.
Jessica
So here you can see similar to the prompts that I just now uh put on Alfie right you can see generate uh APM scripts and what is the expected response.
Jessica
So this is the API response that uh the chatbot or the AI agent evaluation would be expecting right and it would compare that to the actual response.
Jessica
So you would first upload uh the expected responses which post which it would compare it to the actual.
Jessica
So this is the first uh set of data you would need which is the expected response data.
Jessica
And next comes the evaluation criteria.
Jessica
Right.
Jessica
So now coming to our AI agent and I'm coming to the evaluation rules to set what would be the evaluation criteria.
Jessica
So let me just show you how that works.
Jessica
So you can give a name to the criteria.
Jessica
You can create multiple sets of criteria for different models or um AI agents that you would be testing.
Jessica
So here we have created one for specifically for Alfie.
Jessica
Uh we have set the pass threshold.
Jessica
So again it doesn't work on a static pass or fail.
Jessica
You can set the threshold and you will be able to configure this according to the model that you're testing.
Jessica
Right?
Jessica
and what are the key features that it's going to assess on.
Jessica
So firstly validation.
Jessica
So it will just do the basic validation of the whether the chatbot is giving me a response uh whether it's a successful uh response.
Jessica
Next uh the keyword check.
Jessica
So if there are any specific domains a domain specific technical terms that are expected um it will check for that sentiment analysis.
Jessica
So again if what kind of uh whether it needs to be uh whether the tone is positive, negative or neutral uh is it supportive or dismissive.
Jessica
So that is on the sentiment […]
Jessica
Hallucination detection is what um was already elaborated right?
Jessica
So uh supposing if your AI agent makes up some facts or it is not answering and it's adding on more data points than is relevant that would be hallucination detection.
Jessica
uh reasoning if the response is logical and context consistency is if it's able to see the previous chat uh you know the previous history and consistently take that context and give a contextual response.
Jessica
So these are the different parameters that it will mainly analyze on.
Jessica
You can set the thresholds for those as well.
Jessica
So for example hallucination confidence threshold.
Jessica
So only when the LLM is 80% confidence that it's an hallucination will it be marked as a hallucination.
Jessica
So you can set again the these thresholds.
Jessica
Similarly context memory threshold.
Jessica
If it takes about 70% of the previous context then it will be successfully contextual.
Jessica
So again these are fully configurable.
Jessica
We have set it to 8 and 7.
Jessica
You can uh set it as much as you want or as less as you want it.
Jessica
uh scoring.
Jessica
So for different uh parameters that we talked about, right?
Jessica
You can give what is the percentage valid um weight that you want to give for each of the parameters.
Jessica
So for example, if validation is important to you, then you can set this to higher and keyword is less important then you can set that to lower and so on and so forth.
Jessica
So that is the different uh weights of scoring configuration you can give for the different parameters.
Jessica
Um finally is the LLM config.
Jessica
Right?
Jessica
So here we are the AI agent is getting evaluated by LLMs itself.
Jessica
So it's kind of like LLM as a judge and you will see so you can choose which model you want to assess your uh AI agents responses.
Jessica
uh you can set the temperature uh to make sure how deterministic or creative the responses uh the evaluation is and you can give the uh AI agent key right and once that's done you are good to you have set your evaluation criteria and you have your uh expected response data so now we are good to begin our AI agent evaluation so let's get to that so I'm going to go ahead and create a new workflow […]
Jessica
So you can give a workflow name and you can give the endpoint of the chatbot URL.
Jessica
Right?
Jessica
So here is where uh the API is um yeah this is the endpoint URL of the chatbot of Alfie.
Jessica
And then to authenticate the session uh you can give the relevant uh be it authentication token authorization token or header information.
Jessica
So here we're giving the token value.
Jessica
Next, we're giving a template uh for the AI to understand how to fetch the different responses and the prompts that we've given in the CSV data.
Jessica
And then we're going to map it […]
Jessica
And finally, I'm going to go ahead and upload the test data.
Jessica
So, we've uploaded the CSV and now I'm going to save this workflow.
Jessica
So, here is the workflow we've just now created.
Jessica
I can just make sure that it's all good to go.
Jessica
Yeah, the token is there.
Jessica
And now I'm going to go ahead and run it.
Jessica
So the CSV evaluation progress is happening right now and um because we have given four rows uh four different prompts it's going to do it one by one.
Jessica
So the first prompt will get fetched as well as the expected response and then it will hit the API to get the chatbot response right and post which our AI agent AI evaluation agent will compare the two and get an individual scoring for each row and finally after all the rows are processed you will get the overall score of uh this particular workflow.
Jessica
Right?
Jessica
So this will take um some amount of time uh maybe few seconds per step uh post which you'll be able to see the results.
Jessica
So yeah, first row is done.
Jessica
Now the second row is processing so on and so forth.
Jessica
Also each workflow can be run multiple times to get an overall score.
Jessica
Right?
Jessica
So as uh Avinash was mentioning on variance, a one-time run is not sufficient.
Jessica
Uh so you might get a high score once and then the second time there might be different responses and the score might vary.
Jessica
So it's important to ensure that you're running your workflow multiple times uh to ensure that the quality of your bot is consistent.
Jessica
So here we've gotten an overall score of 82.1% uh and all four uh prompts have passed.
Jessica
And now we can get into the detailed overview.
Jessica
So I'm going to go ahead and view the results.
Jessica
So this is the total evaluation of the workflow.
Jessica
So I've just taken a previous workflow which has run nine times and you can see uh the trajectory of the improvement.
Jessica
Uh right you can see the past fail distribution.
Jessica
How many steps failed, how many steps passed, what is the success rate and the overall average score across the nine evaluations.
Jessica
Right?
Jessica
So this will give you a comprehensive view of how your uh chatbot is performing.
Jessica
Next you will get uh the score of the traditional metrics as well as the LLM metrics.
Jessica
Traditional being validation whether your chatbot successfully gave responses uh keyword domain relevant keywords were present or not sentiment um and how fast the chatbot gave the responses.
Jessica
Right.
Jessica
Um now coming to LLM metrics uh firstly it'll assess on relevance.
Jessica
So how relevant was the uh response?
Jessica
So for example in Alfie if I gave something about uh generating uh capabilities for iOS how relevant was the response to that right or available devices uh on iOS did it um hallucinate and give more uh information than was uh required that is what it's going to assess here.
Jessica
Completeness is did it give all the relevant uh data points for it to be a complete answer for the user and uh clarity.
Jessica
How well can the user understand?
Jessica
Is there any ambiguity in the response and hallucination?
Jessica
Has it made any additional u half-truths or incorrect um you know information uh reasoning and context consistency in terms of being able to know the previous um uh context in order to give relevant answers.
Jessica
So that is the overall score.
Jessica
Now I can get into each individual run right and see how well it performed.
Jessica
So you can see I'm just picking out one of the particular runs.
Jessica
This was the overall score.
Jessica
So every individual run will have the same metrics uh and different data points.
Jessica
So you can see uh the overall relevance was 70%, uh 86% in overall hallucination detection and the contextual consistency was 86%.
Jessica
So these are the different kind of uh data points you can get for within every workflow and then within the workflow also you can get the data points for every individual row.
Jessica
So for example here this is the row where uh we have given the prompt of provide the list of available Samsung devices right and this was the expected response that we gave and what was the actual AI response and comparing these two is where these different scores have um you know displayed here.
Jessica
So you can see the relevance was nearly 70% uh with really high uh context consistency and uh clarity and then the final summary is that there were minor or insignificant differences in the details of the Samsung device data but overall the accuracy was high so it's fast.
Jessica
Similarly we can look into some of the failed uh steps as well.
Jessica
So I can look into one of the failed uh rows.
Jessica
So basically this was generate an APM script to test the signup feature and you can see that the there was u missing uh there was some expected content that was uh missing right so the relevance score has come down really low u made it below 70% which was the threshold we had set so it was 66% and hence the status is fail right so again it's not a a deterministic pass or fail so based on the threshold it has determined that this particular um row or prompt has failed.
Jessica
Right?
Jessica
So that is how you would be able to go about assessing your um chatbot and its performance.
Jessica
So you can also go ahead and see all the workflows and then see the overall performance uh trajectory of all the different workflows within your chatbot and get the same kind of evaluation.
Jessica
So I hope this gave you a glimpse about how um our AI agent evaluation works uh be it from setting the evaluation criteria to the kind of results and data you'll be able to see uh on our dashboard.
Jessica
So with that I'll hand it over back to Avinash for uh
Avinash
Uh thanks Jessica thanks for wonderful demonstration.
Avinash
Uh one additional point I wanted to add in the platform is there is a scheduleuler as well.
Avinash
Uh because like we discussed the agents have to be continuously evaluated and monitored.
Avinash
Um so we have built a scheduleuler where you can say keep running uh these evaluations every 10 minutes, 15 minutes, half an hour, one every day.
Avinash
every week depending on your uh uh I mean your use case and how frequently do you would you like to evaluate.
Avinash
So all the data that you saw will be plotted across these different runs which will keep happening depending on how you have scheduled the evaluation.
Avinash
Um so that's pretty much what we have built.
Avinash
So in a nutshell, if you have a agent or an agenti platform, uh you can build the workflows.
Avinash
Uh so what Jessica showed was just one agent, but then in the workflow you can create a sequence of uh APIs as well to mimic the sequence of agent working together.
Avinash
So that's another additional um feature that we have.
Avinash
But the point is that you can configure your evaluation criteria.
Avinash
you can have your test data prepared uh uploaded to the platform and then schedule it and and just [clears throat] look at the evaluation score and if there is a problem you can go and look at the details and figure out where the problem is.
Avinash
So uh that's that's what Pcloudy's agent evaluation platform is.
Avinash
Um just wanted to end that even if you have not started thinking about it.
Avinash
I think like in in overall whatever we discussed that this is the world we are moving towards.
Avinash
So sooner or later you have to decide to learn the skill as well as start building some of these frameworks which might be useful to your organization and use cases.
Avinash
Um so there's no tomorrow.
Avinash
I think we all need to start now.
Avinash
Um and that that is the message I want to leave all all of you with.
Avinash
Uh thank you so much for listening patiently and I'm sure you would have lot of thoughts, feedbacks, maybe learnings.
Avinash
Uh if you have used something like that or built something like that in your organization, we'd love to hear more about it.
Avinash
Um so thank you so much again and would love to u answer any questions that you have.
Avinash
Would love to connect with all of you.
Avinash
So you can just scan the QR code and uh connect to us.
Avinash
Um thank you so much.
Dinakar
All right, great.
Dinakar
Uh thank you so much Avinash.
Dinakar
Just pulling up a few questions.
Dinakar
U let me just see.
Dinakar
Okay, one of the questions was what would you consider essential to evaluate an AI agent like accuracy?
Dinakar
you you had mentioned that in your presentation but
Avinash
we did cover that part and I mean there is there's lot of uh research and work going on to improve these parameters but pretty much those are um at a broad level we cover those areas for example your accuracy your consistency your variance your hallucination etc all those parameters are extremely important um and then we have built our parameters based on those and as new evaluation frameworks are coming up in the market we'll be happy to include those as well.
Avinash
Uh we are continuously looking at newer ways of uh improving the uh or improving the testing or evaluation of these agents.
Avinash
All right, here's another one.
Avinash
um you know how often do you think the AI agent um should be evaluated or re-evaluated?
Avinash
Uh how how long how often should it be retrained is another question.
Avinash
I mean great point.
Avinash
I think that's that's one of the difference um from the traditional testing that we have seen.
Avinash
Um I mean traditional testing is needed when there is a change in the system or if there is a new addition u that has happened in the software but in the case of um agents a continuous testing is the norm right because like I said let's say if you have built an agent over openai or gemini slight change in the update that they do can change the behavior of your agent.
Avinash
So you have to be on top of it.
Avinash
Um even if you're not changing anything on the agent code, uh model changes can impact.
Avinash
So norm is to do continuous testing.
Avinash
Now that continuous frequency depends on the criticality of your use case and some of the other factors but it has to be continuous so that you can monitor the score and see how your agents are either improving or if there are some is there some impact because of some external factor.
Avinash
So until you're doing the test continuously you'll not know and if something goes wrong you'll be caught completely unaware right.
Avinash
So to avoid that situation, continuous testing is the norm.
Avinash
And I think uh uh with a platform like uh Pcloudy that we have it's very easy to just u uh enable the continuous testing by using the scheduleuler or linking with your CI/CD platforms to keep it running and then uh monitor the scores.
Dinakar
Okay.
Dinakar
Yeah, makes sense.
Dinakar
Makes sense.
Dinakar
U this one sai slightly off topic but I think one of the questions was what guardrails must be put in place before an agent is allowed to take multi-step actions unattended or autonomously.
Avinash
Yeah I mean this is a completely different topic and guardrails are one of the uh most frequently talked uh topic today.
Avinash
uh in fact there are companies built uh on on enabling guardrail for agents or AI systems right um and it's extremely important if you are rolling out something in production where your customers are going to use it guardrails are extremely important guardrails could be in terms of maintaining security uh making sure that unauthorized accesses are uh limited I mean there are you would have heard news about prompt injection and all that.
Avinash
So there are guards around guardrails around security.
Avinash
there'll be guardrails around when do you need to intervene as a human uh because I I firmly believe uh there has to be some human in the loop uh even within these autonomous systems right so these systems should know when to um uh send it to a human uh what are the areas where human approval is must so all those guardrails are extremely important when you are building a production system um But that's that's part of the design.
Avinash
But when that those guard guardrails are in place based on those guardrails uh your testing or evaluation criteria will also change.
Avinash
So for example if there are those guards around human in the loop uh you need to design those test data or your u u workflows in a way that uh you simulate those scenarios.
Avinash
Right?
Avinash
So uh it impacts the testing uh criterias or your testing data.
Avinash
Um but guard implementation is more of the design u part which has to be discussed when the AI systems are being built.
Dinakar
Right.
Dinakar
Right.
Dinakar
I think a quick and last one if we have time for just this one last question.
Dinakar
All evaluation is based on the concept of LLM as a judge and uh but what are your references for the ground truth like where is is is there an absolute truth that we provide when we're evaluating?
Avinash
Very interesting question and that's extremely relevant to what we are doing.
Avinash
Um yes LLM as a judge is one of the way to evaluate but uh there are two ways uh to provide the ground truth right one is like Jessica showed you can upload your test data where you provide your prompt and expected response.
Avinash
So expected response is treated as ground truth right and based on that all the parameters are evaluated by an independent LLM as a judge right and that independent LLM could be anything right now we have given different openi models but we are working to integrate other lms as well so it could be gemini it could be claude um so if your chatbot is built on openi you can have claude as your lm as a judge uh if your bot is built on claude you can use openai Right?
Avinash
Or if you built let's say on your custom LLM internal open source LLMs you can use some external third party another open source LM as well.
Avinash
Right now we not supporting the evaluation with external open source LMS but we will bring that capability as well.
Avinash
But yeah I mean the the core I mean the process is to use LLM as a judge.
Avinash
Um but second way is let's say you don't have expected response right now.
Avinash
We just have the test criteria in terms of your prompts.
Avinash
So you can let LLM also decide what should what is the ground truth.
Avinash
So both options are available though Jessica didn't show the second option but it is also there where you can let LLM decide the ground truth and then also judge it based on the based on its own uh parameters.
Avinash
So there are both options.
Avinash
If you don't have the test data, full test data, you can use LMG for both the parts ground truth as well as the evaluation.
Avinash
But we suggest that you create your own ground truth.
Avinash
So let's say you create 100 test data where you you know that these are the expected prompts that I am going to test and these are my expected responses because when you're building uh the AI agent, your developers would also need that test data, right?
Avinash
So that is where I think we have been coining this term like evaluationdriven development right.
Avinash
So first you need to decide what are the evaluation criteria what are the test data and ground truths that you're looking at so that your uh AI teams are using that as part of your development.
Avinash
Sometimes they use it to fine-tune as well.
Avinash
Sometimes they use it to um have let's say few short examples when they build the agents.
Avinash
So that is extremely important to build some ground truth and ground truth comes from your own domain experience uh based on what you want to show to your customer.
Avinash
What is the right outcome you want to present to your customers uh when they use the agents.
Avinash
So our suggestion is to have the ground truth ready and use that ground truth with LM as a judge for a evaluation.
Avinash
But that's the suggestion we have.
Dinakar
All right.
Dinakar
Great.
Dinakar
Well, uh that's most of the questions we've covered.
Dinakar
Uh a few more rolling in, but I think due to the lack of time, I think we'll call it a day.
Dinakar
Uh but I do want to say a big thank you to Avinash and Jessica for sharing your wisdom in the field of agentic AI and evaluating uh you did a deep dive into evaluating AI agents and showing us how to evaluate the agentic AI systems as well.
Dinakar
So, thank you so much.
Dinakar
We also want to thank each one of our participants uh for joining us for this webinar.
Dinakar
We are sure that some of the insights that were shared by Avinash and Jessica may have sparked a few ideas uh that you can think through and implement.
Dinakar
My recommendation would be to mull over these ideas, talk to us to discuss uh and keep the conversation going.
Dinakar
And do feel free to reach out to us with any question like Amin mentioned.
Dinakar
you can just scan the QR code uh leave us a drop us a line and we will respond to your queries as soon as possible.
Dinakar
With that, thank you once again for being an engaging audience and uh happy evaluating.
Avinash
Thank you so much.
Avinash
Thanks everyone.
Avinash
Have a good day.
