Dinakar
We will answer your questions as part of the Q&A section. We have a dedicated time slot for that towards the end of the webinar. So keep those questions coming and if we do have time to take them, we'll definitely answer them online. else we'll we'll answer them offline. And also please note that this webinar is being recorded and we'll send out the link of the recording to share with your colleagues or to watch it again. so with that said, let me introduce the speakers for today. I mean we've got Jessica. So Jessica is a test evangelist at Pcloudy. she has
Dinakar
Around four plus years of experience in the IT industry. prior to her role here at Pcloudy, she has worked with LTIMindtree as a pre-sales consultant. she used to design automation solution design for various industries and she has a keen expertise in data analytics and various automation tools. so without further ado, let me go ahead and kickstart. we'll do things a bit differently today. I'll start off the webinar by doing a few slides and then I'll hand it to our speaker Jessica.
Dinakar
So I'll call her out and we'll go from there. Right. So just as we start there are different things when we talk about test suites right when we talk about test suites we really talking about the foundation of quality assurance right test suites are not just the checklist they're what keeps the product stable as it grows. for example a solid test suite ensures that your core features don't break. they make it regression proof. It also acts as a live documentation. So when you have a good test suite it reduces
Dinakar
The manual load of your retention on testers. U the retesting efforts that the testers have to take it kind of reduces that so the manual load of your testers is reduced. It also enables true continuous integration allowing you to deploy your releases with confidence and more importantly your test suite acts as a protective mechanism during your releases.
Dinakar
It provides a safety net for catching issues before they even hit production. So that's some of the key roles that we kind of discussed when we're talking about a test suite. Now when we look at what makes a good test suite firstly we must remember okay great firstly we must remember that it covers the critical user journeys that your you to kind of monitor the things your customers will actually rely on and second it should be low on maintenance if your team is constantly fixing
Dinakar
Broken tests then it's a liability it's not an asset it's not going to be helpful full in the long run, right? So, a good test suite should have should be low on maintenance without having your team to go back and constantly fix tests there. third one is, clarity is key. each step should be, easily understandable, something that you can reuse later on, especially when you have multiple teams involved and multiple projects that where you have a cross-section of teams working on different projects. so when you build a test suite make sure that it's having some clarity so that you can go back to and reuse some of those scripts right
Dinakar
A test suite should also be stable across environments it shouldn't be that it just passes in the staging environment and then fails in the production environment but for any reason it should kind of be stable across environments so that you've got all you've got no noise but just signal Right. and finally, a good test suite is easy to grow, something that you want to refactor, something that you want to reuse, that you want to add on and reorganize without starting from scratch all the time. So, with that, let's quickly go and look at a poll. We've looked at
Dinakar
What makes a good test suite. We've looked at, how to make a good test suite. and here's a quick poll that to understand the kind of audience that we have here. so I mean if you have the poll questions you can display them. So go ahead and start filling up the poll. The question is are you spending more time writing tests than actually executing the tests?
Dinakar
I'm going to keep that open for some more time. We have a lot of votes coming in. mostly a yes and sometimes I'm going to give it 30 more seconds for you all to fill up that poll. Are you spending more time writing the tests than actually executing it? That's the question you're polling for.
Dinakar
Okay, I can see the sometimes moving a lot more, but I mean I'm going to wait a few more seconds for you to close in your votes. Okay. Wow. Okay. Okay. It's surprising to see that nobody has a no there.
Dinakar
So you see the sometimes and okay I can see that so I mean if you're able to see the percentages on display you can see that mostly 61% of the crowd that's here is saying you spend a lot of time writing test than actually executing the test. So sometimes you kind of do that and a yeses are a bit on the a yeses are a significant percentage as well. So 40% close to 40% is saying yes they spend a lot of time writing test than actually testing it. So let's see what we have to learn today. so with that let me hand it to our speaker Jessica Selvin who will take the presentation
Dinakar
On. So over to you Jessica.
Jessica Selvin
Thank you. Thank you so much Dinakar for setting the context and giving us an idea about the importance and the characteristics of test suites as well as making it interactive with this poll. so with that context let's just look into some of the other current approaches that we see with automation today right so and as testers you might already be aware of this but just to give you an idea right like firstly what we do is we gather requirements be it testers or automation engineers they gather the requirements be it from multiple sources
Jessica Selvin
Maybe user stories BRDs from the manual test cases And this step would typically involve quite a bit of back and forth between your devs, business analysts or product owners. right and next comes the task of the test script creation. right so be it whichever framework that you're automating in be it selenium Appium Playwright right and this again is done step by step manually and can be time-consuming right as was attested by the poll and next comes the verification and the validation of the
Jessica Selvin
Tests so maybe debugging the locators adjusting wait times fixing test data issues this itself can be a pretty highly time consuming task and finally our text test execution where we maybe execute it locally first and then push it to the CI/CD for your broader regression runs right so for example in a real world scenario it would be say you're testing the login feature for a banking app using Appium you'd first read through the user story and or the Figma to understand what are the login conditions then you would manually write the Appium or the
Jessica Selvin
Selenium test with the locators, do an inspect identify the locators, see which is the best fit locator in order to incorporate in your script. right and maybe you might face some flakiness due to any OTP scenarios or complex scenarios or edge cases and then finally you run it on your devices and log the results.
Jessica Selvin
So this is the current state of script writing that's happening. That is the process. Now we're going to look at some of the challenging realities that is very much part of the day in day out of testing. Right. so beyond the process right what are the some of the real challenges that is seen? firstly it is time consuming right writing even one good end-to-end test scenario can take hours especially if you're building for scale if you're building for cross-platform scenarios it can take a huge amount
Jessica Selvin
Of time next automation scripts can be fragile right so test break with UI changes there could be some dynamic locators synchronous weights or just a change in the text right and sometimes even following certain best practices, flakiness can happen in your test cases, right? And next is the manual testing lacks coverage, right? So manual QA can often miss edge cases not because testers aren't good but simply because the scope is simply too wide and time is limited particularly with the agile framework of sprints releasing new
Jessica Selvin
Features on a weekly sometimes daily basis even right and next maintenance can eat up the velocity right regression cycles can become a bottleneck and you end up fixing tests more than writing new ones and automation can become a burden of itself, right? And this keeps your test debt growing. So new features keep coming in but the automation can't keep up and teams are pushing it to the next sprint and that's how your test debt can keep piling up. So these are some of the realities that is seen in script creation for testing
Jessica Selvin
And in addition to that right what are the challenges in applying or implementing test automation all right so as I mentioned in the agile framework the product keeps changing specifications keep changing mid-sprint and test cases new features are added right and some test cases can become obsolete pretty quickly. And secondly, lack of qualified resources be it in terms of time, skills. right, it's not just about knowing a tool. You need to understand the product, the frameworks, the data flows. So it
Jessica Selvin
Requires a lot of diverse skills and resources. which could be also a limiting challenge. And next is difficult to integrate with other tools. So there might be some legacy tools isolated CI/CD pipelines without test orchestration that has an end to-end flow right or lack of integration with your Jira or your test management. this can also ensure that certain test cases and certain checks fall through the cracks.
Jessica Selvin
And next is there are no right processes or methods. So there could be a lack of scope understanding ownership strategy which ensures that which can lead to just partial coverage of your test right of your product and finally test data and environment challenges. So a test case can fail not just because of bad scripts but maybe test data issues or the wrong user state or test environments that aren't aligned with production. right. So what we can conclude here is some of the true
Jessica Selvin
Blockers in script automation scripting are not just technical they're human they're process related and they're ecosystem and environment related as well. So all these things can play a factor in how you are applying test automation in your day-to-day and some of the other challenges that we see is firstly automation can sometimes become slow to scale up right you can start with a few automation test cases it feels manageable but as your app grows the test scripts it's very difficult to
Jessica Selvin
Scale up fast right especally especially sometimes with limited number of team members right so it's difficult to scale up at times next there is a high maintenance time and efforts that are required right so tests break not just because the features are broken but sometimes the scripts itself can be brittle there can be any UI change or ID change that can cause this right and someone might have to manually test this and then maintenance becomes like a full-time job.
Jessica Selvin
And human error can always happen in the loop, right? Which also has to be taken into account. And as I mentioned, there can be brittle UI changes. Test scripts are tightly coupled to the UI structure. So if the devs change anything in the DOM, then the script itself can collapse, right? And because of all of this, there is a lack of time in order to ensure full test coverage, right? and edge cases, negative flows and boundary conditions are often left out right limiting the test coverage.
Jessica Selvin
But there is a shift that is happening in our in the quality engineering right some of the shifts and trends that we see right so firstly from manualheavy to intelligence assisted so some of the old way could have been manual testers would execute test cases one by one log the results in Excel or Jira and then write the reports manually now what's happening is AI tools assist the testers in auto-generating the test cases auto-filling test data validating results and even suggesting edge test cases and these systems can learn from past behavior and
Jessica Selvin
Help guide decisions and they become like co-pilots. Next from scripting to agent guided flows right so the old way was maybe writing each step-by-step automation scripts line by line in using Java or Python JavaScript with the framework right like Selenium or Appium right and now what happens is and now the trends that and the shift that is happening right now is that agentic platforms ask you to describe the scenario in natural language and then they are firstly able to understand the goal identify the UI elements, generate a
Jessica Selvin
Test flow and also handle scenarios like wait times clicks selecting from drop-downs toggles inputting data as well as validations and assertions. right next from static testing plans to adaptive coverage. So the old way might have been you write a fixed test plan and with predefined test cases and exit sprint right and now what we are seeing is test coverage to adapt dynamically to code changes recent bugs as well as journeys from the production logs right fourthly from
Jessica Selvin
Automation tools to orchestration agents right so having to coordinate between your CI/CD your environment ment maybe your cloud setup and then your test runs or your local device setup wherever you're executing your test cases right and then handling reporting maybe in a separate tool right all these are disjointed fragmented tools to now orchestration agents right which are able to where you're able to select generate your test cases then select the test cases test suites set up the test data and then trigger the runs across your cloud platform right and retry try maybe regenerate the failed test
Jessica Selvin
Cases auto send reports or log bugs in for fail test cases and get the full-fledged automation reports. Entire flow and orchestration agents are coming into play here and finally from backlog to real-time execution.
Jessica Selvin
So test cases are always behind where devs are finishing a feature and testing lags by one to two sprints and that's how test debt happens. But now what we're seeing is as soon as a user story is created or a UI screen is updated agentic AI is able to create test cases execute them on staging validate results and push issues into backlog or resolve them if it's fixed. So all this is happening even before maybe the app comes into the testing stage itself. So these are the some of the key shifts that are happening and that are observed
Jessica Selvin
In QA. So yeah that is the big message that we're driving at here is to adopt agentic AI and that is the trend that's the future that we all need to jump into and adapt ourselves into. So to see the journey and how it has happened right. So firstly how testing started manually with Excel reports with manual test cases writing everything by hand tracking the defects in Excel sheets. it's a working system but it's completely impractical to scale. right
Jessica Selvin
Next is the waterfall methodology where testing came last after the full development of the build after UAT. And the issues were found late which generally does incur a higher cost right and testing can be slow it's reactive rather than proactive and then the age of test automation where selenium came your Appium scripts right where we began automation test cases and regression testing but it still required scripting skills and high maintenance especially when agile and CI/CD came into the picture, right?
Jessica Selvin
Testing shifted left and automation had to keep up really fast with the weekly releases and even daily releases at times, right? And AIdriven testing is something we started seeing in the recent past with AI augment testing like auto identifying elements classifying failures or suggesting test cases. right and now in today's age it's not the future. It's already here.
Jessica Selvin
Right we are entering into the era of AI agents these are intelligent systems that don't just assist they plan create and run analyze and even selfheal your tests so it's no longer just automation it's autonomy right where there is still of course human intervention but there is an end-to-end autonomous flow that's able to take care right from your test generation execution and results so Yeah, again driving home the point that AI agents are here to stay and will lead the way in autonomous testing.
Jessica Selvin
So what are AI agents? To define it, it is a software component that has the agency to act on behalf of a user or a system to perform tasks. So in simple terms, it is goal oriented. So it is not just a simple like what you might with generative AI. it has a goal oriented nature where it's able to perform the task right and accomplish it and not just respond to a prompt.
Jessica Selvin
So just taking a step back from AI to introduce Pcloudy at this point and how our AI agent is fitting into the whole testing scenario right as I mentioned right orchestration is coming into the picture where we don't just have fragmented tools right we need to incorporate it into the larger framework of how testing works right so with that introducing Pcloudy our AI powered digital experience testing platform right. And the core of our platform is our diverse device coverage where we have thousands of devices and
Jessica Selvin
Browsers combinations right real devices that are hosted in our data centers right physical devices that are hosted across in multiple locations in geographic geographical locations and we also support custom devices. We have several deployment options of on-premise lab in a box which is a quick to set up on-premise option as well as private cloud which is exclusive set of cloud devices for you to test with and our public cloud right which is a sharable platform with a vast range of devices.
Jessica Selvin
And the first layer of our solution is our functional experience testing. where be it you can simply connect to a device of your choice. install your developer builds right and go ahead and begin your validations your manual testing as well as your automation testing. So we support integration with your traditional automation frameworks as well. So all your existing Appium, Selenium, Playwright and typically any other popular automation framework out there, right?
Jessica Selvin
We pretty much integrate with that and you'll be able to execute that on our platform and generate results. So that is something that we support along with specialized features. Next, another interesting space that we have moved into is our performance experience testing module where we are tracking the digital experience of the users. So we track 60 plus metrics at the UI level at the individual app level and user level and we're tracking the backend metrics like the network and API calls that happen behind the scenes right as well as your different performance
Jessica Selvin
Metrics like your CPU memory battery right so 60 plus metrics in a comprehensive dashboard visualizations with drill downs and you'll be able to analyze the digital experience of your application And next another space that we are moving into is synthetic monitoring which is live monitoring of your production applications right and you'll be able to run your critical workflows 247 and get the performance insights your anomalies and trends on it. And with all these solutions our AI agents are driving it. Right. And the
Jessica Selvin
Most important one that we're going to be talking about today is our Qpilot agent which helps right from your test script generation triggering your execution and generating reports. And then additionally we also have self-healing and orchestration agents and observability agents in order to generate performance insights. So that is a brief overview on our entire platform. with this we're going to be deep diving into Qpilot, but if you're interested in any of these other features, right or capabilities that we bring in, feel free to reach out to us.
Jessica Selvin
So, yeah, introducing our AI agent Qpilot. it is a platform agnostic autonomous AI agent, right? So, it is a goal-driven AI workflow for performing testing tasks. So it works across your mobile apps, your web apps. right, we're technology agnostic, right? So different type of applications, your flutter applications, your hybrid native apps. it works on that seamlessly. there are no additional prerequisites for any iOS based applications. It is very much a platform agnostic AI agent, right? And how is it different from your
Jessica Selvin
Genai tools like chat GPT? Gen AI responds to single prompts but doesn't ensure goal completion here with a specific goal right it plans out the workflow and executes the tasks autonomously and it's specially designed for testing scenarios right so that is how we differentiate from your typical generative AI agents so how does the entire workflow happen with Qpilot right so Just to give you a bit of how the flow works.
Jessica Selvin
Firstly, we start with a natural language input. It starts with plain English. right, not code. and the only inputs required is a natural language description of your test scenario. So, right, there are no keywords or a specific format in which you need to give this as you will see in the demo. Next is the interpretation and reasoning. So, Qpilot uses NLP and AI reasoning to understand the intent and break it into test objectives and map them to UI elements.
Jessica Selvin
So, for example, if you're specifying username field, it'll go through the entire DOM structure, try to find the closest match to that and do the particular action, right? To identify the closest match to those UI elements. Next is the workflow generation where it generates a full workflow script for that particular step. So it is going to fetch the XPath the locators and try to see the best fit that script for that particular step and post which it will execute it. So this workflow generation and execution happens in multiple iterations to
Jessica Selvin
Ensure that the final result and the outcome that you see is highly accurate. so there is a lot of learning and course correction that happens behind the scenes during the phase of script generation. And finally we get the output or the outcome which where you'll be able to see the fully generated ready to run script. So you don't need to spend time debugging in like you would see in typical other AI tools. there would be a lot of time spent in debugging the script which would again require coding knowledge right so and is time-consuming so our scripts
Jessica Selvin
Once generated are fully accurate and ready to run and some of the tech that's happening behind the scenes with Qpilot right as I mentioned it uses a agentic AI and there are multiple agents that are happening that are working behind the scenes in order to perform the task. So firstly, it is the XPath or the locator generation agent. right, where it's responsible for finding the right way to identify the UI elements.
Jessica Selvin
So be it buttons, check boxes, drop downs, input fields, right? It's trying to it's responsible to see the right way to identify it, right? And how it works is firstly it passes through the HTML or the app's UI structure, the entire DOM structure, right? the hierarchy for mobile apps or the your web apps and it looks at the ids, the names, the classes right and the text content positions right and it's able to choose the most stable and the reliable locator right and next what it does is it's able to handle avoid brittle locators and see the best fit
Jessica Selvin
Locators as well as have a fall back and a retry strategy. So if one locator fails at runtime the agent can fall back to alternate locators to ensure that the script is more robust. And finally the other agents that come into play are the code generation agent that goes ahead and generates the code and the step execution agent that executes it step by step. And between these two agents is where the feedback mechanism happens as I mentioned. And finally if in case there's an error also to give what is the exact error that happened or what
Jessica Selvin
Was the reason for the script failure script generation failure right so these are the different agents that are working behind the scenes so with that context given let's get into how Qpilot works and let's get into the live demo right so just give me a second to share my screen again.
Jessica Selvin
Yeah. So this platform that you see right here is our Pcloudy platform. right it's a SaaS-based platform and it can be accessed very simply through registering and going ahead and beginning your usage be it connecting to our devices and browsers. And as I'm demonstrating it on our public cloud platform, if you're interested in a trial or would like to know more about our entire platform, please feel free to reach out to us.
Jessica Selvin
So with that, I'm just going to go ahead and connect to some or see first show our devices and then we'll go ahead and connect to our device to begin our script generation. So as you can see and as I mentioned right, these are all real devices. physical devices hosted across in our data centers and you will be able to select from a vast range right so we have all the latest OS versions like Android 16 as well as the older versions and similarly on the iOS side we have iOS 26 18.5 and the older iOS versions similarly we have all the different OEMs like Apple, Google
Jessica Selvin
Samsung and all the different popular OEMs. You can also filter out by screen size. We have SIM card enabled devices where a SIM is inserted to the device. So you'll also be able to do OTP based verifications, SMS verifications and so on. And as I mentioned, we have multiple geographical locations. several in India as well as the US, Singapore and the UAE. So this is just to give you an idea of the level of device coverage that we have.
Jessica Selvin
With that you can just simply connect to a device of your choice. So I'm just clicking on connect. As soon as you click on connect right the device session request is sent and there will be a session established right that is exclusively available to you. So there is no session time limit as such to the amount of time you can have with the device. So now our session is established. It's pretty much like having a physical device in your hands. you can navigate it seamlessly. And there are several features in the manual mode itself. if you're interested, you can reach out to us, right? Where you'll be
Jessica Selvin
Able to test for accessibility, logs, screenshots, network location simulation, QR code scanning, biometrics and so on. So we have several specialized features in the map with but our focus of the discussion is Qpilot. So I'm going straight in the Q mode.
Jessica Selvin
So firstly you will simply select the application from the drop down right. So yeah I must mention this before you connect to the device you simply upload your developer builds your or your test data right your APKs your IPA file into this section right and once you do that you can again come back connect to the device and begin your testing so I'm just selecting a sample banking app build you can give the app package and the app activity name can tag it to a project
Jessica Selvin
And give a feature name and then simply give the natural language description of your test case. So I just have a scenario here which is again in clear natural language. I have in case if you do not want to input stat static data your username or password you can add it as test data. So that's what I've done here and you can give it in the format of give it in JSON format. So the username password and name or any other particular field that you don't want to hardcode you can put it in the JSON
Jessica Selvin
Format. Click on add test data. So this is our natural language here. And once that's done, the description is given. You can go ahead and add it, right? Create a test suite. And I'm just going to give it as demo. Give a test case name and save it. Now once that's done, we are ready to go ahead and generate our test case.
Jessica Selvin
So as soon as you click on generate firstly the app that we have given here will get fetched from the my data section and installed into the device post which what will happen is the AI models will kick in and try to go line by line and understand each step and as I mentioned right the XPath locator agent will come into play go through the entire it will pass through the entire DOM structure or the hierarch of your application Okay. And then it will analyze see what is the best fit action to be done or clicks to be done.
Jessica Selvin
And I have just given a fairly short script here but you can go ahead and give any number of lines. We do not have any limitation as such to the number of steps that can be for a particular test case. And some the so you will see that some of the steps do take its time right on average sometimes it may take between five to 15 seconds per step and that is intentional in order to have higher accuracy of the script right so that there's a trade-off there and we have chosen not to compromise on accuracy so though the first time execution will take maybe
Jessica Selvin
A little more than 5 seconds the sum will be much faster right And but the advantage here is that you're going to see a higher accuracy in your scripts. You don't need to spend time debugging your scripts. So yeah, we are doing a simple flow of transfer in the sample banking app, right? And also as you can see I have added some validation steps. You can add validation steps. You can do slide to left actions. You can do click on device back buttons. Wait time. You can also steps of like capture screenshots. So these are the different types of
Jessica Selvin
Scenarios that you'll be able to and test descriptions you'll be able to give for the AI model to understand right and particularly mobile scenarios we are a key differentiator where we are able to achieve some of those kind of test cases as well like I mentioned right like being slightly left or clicking on buttons and these are some of the some challenges that typical AI models would face where we are able to have the advantage. So our test cases is fully generated now and you can see that it's redirecting us to the Qpilot dashboard. So let's get to the cuppilot dashboard and you'll be able to
Jessica Selvin
See all your test cases here in u our cupillet dashboard your web yours and Android all your test cases will be over here. So before we go ahead and execute the script that we have just now generated let me create one more for web as well. So this is our browser infrastructure.
Jessica Selvin
We have our Windows and Mac machines. you can similar to how I showed you with the mobile, right? You can go ahead and connect to a browser machine, right? So again a browser session will be launched which will be exclusive for you. And that will first be in the manual mode post which you can click on the Qpilot icon to enter into the Qpilot mode. And in the for web test cases you do not need to you simply need to give the URL and then the subsequent test case description and then you're good to go.
Jessica Selvin
Let's try another browser. So again you can achieve cross browser testing here as you can see with multiple different browsers and different versions. So yeah our browser is now connected. We can simply get into the Q- pilot mode.
Jessica Selvin
So, and then you can just simply give the URL that you want to test for. So, I'm just giving the URL. You can tag it to a project, give a feature name, and then the test description, right? So, I have a test description out here. So, this is a pet food retail website, right? where we're going to do the action of adding an item to the cart.
Jessica Selvin
So, I'm adding it to the same suite. right. And I'm giving a feature test case name. Now, we're ready to go ahead and generate our script. So, the generation has started. firstly the URL will get launched in the browser and as I mentioned then the AI agents will kick in where it will pass through the entire DOM structure and hierarchy of your application and then try to see the best
Jessica Selvin
Fit match and what are the clicks to be done. So firstly yeah the first step is already done post which it will subsequently each step will first the script is getting generated and then as you can see the execution is also happening to ensure that the script that is the step that is generated is accurate right so let's just wait for the workflow to be completed again even for web you'll be able to add test data you can also add API steps in cases where you need to add any undeveloped feature which is still in the form of a curl API for example you can to perform that particular action you can also add an API step right so these are
Jessica Selvin
Some of the features that we have so yeah our script is almost done yeah so you can see that in a matter of less than a minute right we've been able to generate a full workflow imagine the time that it would take in order to say are doing a selenium script for this, right? You need to inspect the elements, see the best fit elements and then generate the script, right? So, this is a massive time save from what you can see, right? Again, it's redirecting us to the Qpilot dashboard. So, now we'll be able to see both the test cases that
Jessica Selvin
We have generated just in a matter of few minutes, right? So, this is the power of Qpilot where you'll be able to now generate full suites, right? so these are the kind of test cases you'll be able to generate across your iOS. iOS works exactly like similar to how I showed you with Android. It works pretty much the same.
Jessica Selvin
So be it your Android, iOS, your web use cases, all of that can be will be visible here in your cuppilot dashboard. so let's get into the execution part. you can yeah so let's first see how the script looks like the generated script you will have access to see the entire script right and as you can see for every field the corresponding XPath locators multiple locators are also there as I mentioned with the fallback or the retry locator also being present right so you can see the script generation and we also have
Jessica Selvin
Options to generate in JavaScript as well as Python And you can also modify the environment. So say for example this particular script holds true for a new build as well. you don't need to generate all your test cases again. So for the older features or the existing features you can simply go ahead and edit the test config, select the version three of the application maybe right and go ahead and do that. And similarly you can modify the feature name, activity, package name as well if that changes in the new build. and then you can simply go ahead and click on run.
Jessica Selvin
You can also modify the test data here. So the test data is visible here. You can save new test data and run it for that the other scenarios as well. You can run as an individual script or you can add it to a collection and execute it as a test suite. So let's see how that happens. So all your test scenarios will be visible here all the test cases that you have generated so far and you can simply go ahead and insert all your test cases right so I'm selecting say some of the test cases we generated today right are
Jessica Selvin
One from Android and a web test case as well as an iOS based test cases right so I've inserted all of that and then I can give a name to my collection And give a name to the project and then you can select the device range that you wanted to execute on. So say for example I wanted to execute on the Android test cases. So as you can see we
Jessica Selvin
Have an Android, web and iOS. So we need to choose the corresponding devices to execute it and it'll execute it on Pcloudy's device itself. So this is how as you can see it becomes an end-to-end auto automation solution for you where you don't need to do a separate setup right be it in terms of your frameworks or be it in terms of devices all of that is available in a single platform here right so I'm choosing one Samsung device and the other maybe I wanted to execute on a Google device and you can also give the version range right so say for example between version 12 to 15 similarly on the iOS as well I can give a version
Jessica Selvin
Range. And I'm selecting a device. And then again Mac. And I can select it for multiple browsers. I'm just giving one here. And then save it. So this is the all the collections that you have here. You can simply go ahead and click on run and trigger all the test suites here. So I'm just showing it to you for three test cases in the interest of time. But there is no limit here. hundreds of test cases can be executed in a single shot. So there is no limit to the number of test case counts as such. And we also have a curl command which you can paste to
Jessica Selvin
Your CI/CD and execute the collections from your CI/CD tools as well. I'm going to go ahead and run it from Pcloudy UI itself. So you can see that the collection execution has started. So the request has been sent and as the sessions become live it will get reflected in the live view session over here. So it'll take maybe a couple of just a bit of time for it to start reflecting in the live view session and as and when the device sessions are getting established you
Jessica Selvin
Will see the live happening here. So yeah, you can see already the browser automation has started. so these are the manual sessions that we connected in order to generate the test cases but this one is the automation that we have just now selected. You can click on the live view in order to see the live execution happening as well. So you can see that the URL is getting launched and the automation is starting and again two other automation Appium sessions have started one for an iOS device and one for an Android device and both are ongoing as well. We can go ahead and see
Jessica Selvin
The live view sessions for both. So you can see this automation is currently in progress. we can also see the one for Android and we can see the one for iOS as well. again if you do not want to track via live view you can also track it through our live reporting we have our progressive reports here where you'll be able to track all the automation runs. You can see that all the latest automation runs that have been picked up. So you can see this live view is currently happening in the Android and is also reflected in the live reporting.
Jessica Selvin
Similarly on the iOS as well. So let's just get into our PR reports. So this is the execution that's currently happening. You can see all the test cases that have passed. right. So the browser use case is already passed and you'll be able to see the video of the entire session. You'll be able to see when was the last updation, the status of the test case as well as the text logs, right? So, you'll be able to see the output of each and every step. If you had enabled it for screenshots or if there was any step that failed, there will be a screenshot also over here present to
Jessica Selvin
See what could have gone wrong. So, that is something that you'll get with the browser reports. Also, these links are entirely sharable. So, you can share it across teams. So the person need not have pecloudy access as such in order to view the reports. Similarly we also have let me just pick up a mobile test case as well. So this has also passed. So you'll be able to see the video of the entire session. All right along with the text logs the device Appium logs as well as the app profiling. So this data might take a couple of more minutes in order to get populated but it'll get populated
Jessica Selvin
Eventually. And again this is also can be shared across seamlessly. So this is how you'll be able to see the status of all your test cases. right and you'll be able to easily track it, easily share it and debug it as well with the Appium logs, text logs and the video. So this is something that's generated by default for every automation run. So I hope this gives you a clarity on how Pcloudy and Qpilot can give you that end to end autonomous testing journey for you right from your test script generation
Jessica Selvin
To executing it seamlessly on your choice of devices and generating reports. so with that we'll open it up for Q&A now and we can take it up from there. So let's see.
Jessica Selvin
Yeah, I'm just looking at the questions. What happens when few things are changed on the app? Will the agent adapt the changes to the test suite? Yes. Right. So firstly to we need to have a restructuring of how we think about this. Right. So with the AI agent, it's just a matter of regenerating. Right. as I mentioned, we have the test config option where you will be able to say your new build comes which has maybe there might be some changes in the XPath or the locators, right? You simply click on generate again and it will or again
Jessica Selvin
Go pass through the entire application and it will populate it again with the updated locators. So you don't need to do that inspecting. You don't need to manually change the script. It will simply go ahead and fix it by you by regenerating the script. Can we integrate this with our existing CI/CD pipelines or test scripts? So you we do have an option to copy paste the script. You can copy paste the script onto your existing frameworks but you might have to do some changes here and there right it may not be as seamless when from doing it end to end with Pcloudy itself but yeah by
Jessica Selvin
And large the test script will remain same the XPath locator information all of that will be there intact right so you can reuse the script in your own framework book test description. I missed any steps. Does AI give any information on this?
Jessica Selvin
Okay, I think the question you're trying to ask is if there's any incomplete test description, right? So yes eventually that's where AI will go into where you simply give a highle description and it's able to go ahead and generate the scenarios being able to intuitively do the actions but as of now we do it step by step in order to ensure accuracy. So there might be many AI agents that out there that are promising with a single line it'll generate everything but the reality is that as of now it still does that will compromise heavily on accuracy right so
Jessica Selvin
Right now we do it step by step but eventually yes that's where Qpilot will be heading to where we'll be able to it'll pick up on missed information as well but as of now yes even if there are say for example you make spelling errors or you do not give the complete name of a particular object. It does have the intuition in order to click on the best fit.
Jessica Selvin
So you don't need to in case of spelling errors or income like partial description of a particular field name right it will still be able to adapt and do those particular clicks. Will this work with edge cases or only standard flows? Right. Yeah. I mean let's have a discussion on it. There might be some cases where we might need to look into what is the exact test case about but we pilot has been able to handle some of the complicated flows as well right like recently we've been able to achieve it
Jessica Selvin
On more complicated applications as well. So please let us know your use cases we'd be happy to work with you in order to come to a solution. what is the learning curve to adopt this approach? Yeah, so I mean that depends right. So if you already have your existing frameworks, we are not saying to dismantle that. That is not the solution. Q in it will work in tandem with the automation frameworks that you already have in place.
Jessica Selvin
So as I mentioned, you can copy paste and use that to your existing scripts. And in our road map we do have a solution where you will be able to select the framework. So for example if you're using Playwright or if you're using Appium you can select that particular framework and generate it exactly to that so that you can reuse it within your own frameworks as well.
Jessica Selvin
How do Pcloudy applications execute applications that are hosted locally? Yes. So we have a feature called wnet. So even if your application is accessible only within your firewall right even in those cases it will be able to go ahead and we have a while feature where through tunneling you will be able to go ahead and access those local applications. Several of our enterprise customers are using this feature. right, this is something that we provide u as a very basic feature and most of our enterprise customers are using this. So that should not be an issue at all. right
Jessica Selvin
It's just a matter of configuring the wnet and you are good to go to test your local applications. We also have detailed documentation on this. or you can just simply reach out to us and we'll be happy to give you a demo on this. is Qpilot a usage of MCPS? I need to look into this. I'm not sure as of now, but I would need to look into this and check with the product team on it.
Jessica Selvin
I think we're good with most of the questions. If there are any questions that I missed, you can maybe raise your hand or ping it again. But I think we're good with most of the questions. So with that I hand it over to Dinakar.
Jessica Selvin
Yeah.
Dinakar
Great. well thank you so much Jessica. I mean thank you for sharing your wisdom in the field of test suites and agentic AI. you're doing the deep dive into building test suites with AI agents and showing us how AI agents can assist with building these reusable test suites. So thank you. we also want to thank each one of our participants. You've been a great engage engaging bunch. so we want to thank you for joining the webinar and keeping it very engaging with your questions and u all the it shows that you were listening keen very keenly and looking at all
Dinakar
The things that were being displayed here. and we're sure that some of the insights that was shared by Jessica may have sparked a few ideas to think through and implement. my recommendation would be to malo these ideas talk to your teams reach out to us if you'd like to discuss further and let's keep the conversation going. and also please feel free to reach out to us with any questions. we've got our email ID as info@Pcloudy.com or supportpcloudy.com or you can even click on the chat option on our website Pcloudy www.Pcloudy.com pcludy.com just go in there and hit the
Dinakar
Chat icon and you'll you'll be directed to somebody and your questions will be answered from there as well or you can reach out to us on LinkedIn you can reach out to us on our handles our names are we've got Jessica Selvin and I'm Rya and you can you'll find us on our on our LinkedIn page as well on PC Cloud's LinkedIn page so if you're not following us do give us a like and a follow there. and I mean you always have the recording. We'll shoot out a recording of this webinar over email and you can reach out to us by responding to that email as well. So
Dinakar
Thank you so much for thank you everyone for being such a great bunch. thank you once again for being an engaging audience and happy testing.
Jessica Selvin
Thank you.
