Dinakar
Alrighty.
Dinakar
It's so nice to see everybody who's joined in here uh early.
Dinakar
Uh we're going to start in a couple more minutes.
Dinakar
Just wait with us while others join in.
Dinakar
But we really want we want to appreciate your time and we appreciate the guys who have joined in right on the dot.
Dinakar
Uh but we will be starting the webinar shortly in a couple more minutes.
Dinakar
All right, great.
Dinakar
Uh, good morning, good evening, good afternoon to everybody joining us from different parts of the world.
Dinakar
Uh, my name is Danaka and I'll be the host for today and I'd like to take this time to welcome you to Pcloudy's webinar on mastering test orchestration.
Dinakar
Learn how AI agents reduce the execution time by 70% and improve your […]
Dinakar
So before we get started, I would like to go over a few housekeeping rules.
Dinakar
You may have noticed that your line is currently muted and uh if you have any questions, you can submit your questions during the webinar using the Q&A option that is available.
Dinakar
Uh we will answer your questions as part of the Q&A section at the end of the webinar.
Dinakar
So please do keep those questions coming in.
Dinakar
We'd love to love for it to be very interactive.
Dinakar
And also please do note that this webinar is being recorded and we will shoot out the recording the link of the recording to share with your colleagues later or to watch it a second time.
Dinakar
With that said, let me introduce the speaker for today.
Dinakar
We've got Jessica Selvin.
Dinakar
Jessica is a test evangelist at Pcloudy.
Dinakar
She has around five years of experience in the IT industry and prior to her role at Pcloudy, she's been working with LTI Minry as a pre-sales consultant.
Dinakar
Uh she was doing this in the automation solution domain uh for various different industry and uh she has been a keen exper she has a keen expertise in data analytics and various automation tools.
Dinakar
So without further ado, let me hand it to our speaker for today Jessica Selvin.
Dinakar
Over to you.
Jessica Selvin
Yeah, thank you Takar for that introduction.
Jessica Selvin
Um, so yeah, askar has already highlighted today we will be talking about uh test orchestration and how uh orchestration can bring about reduction in your execution time by 70% improving the stability, reliability and efficiency of your test suites.
Jessica Selvin
Right?
Jessica Selvin
So with that let me get started.
Jessica Selvin
Yeah.
Jessica Selvin
So firstly let's take a step back and look at the reality of today's QA world right um where and let's see where are you know we face the hiccups uh and what are the new trends that are arising firstly we see that there are much shorter release cycles now right so our time budget for quality is collapsing so when you ship features so quickly maybe weekly or daily there isn't room to run a 10-hour regression or wait overnight to learn what are the critical flows that broke, right?
Jessica Selvin
Uh modern CI/CD means that every commit is potentially releasable.
Jessica Selvin
So tests have to provide the signals immediately.
Jessica Selvin
So that pressure exposes every inefficiency in the way we schedule and execute tests.
Jessica Selvin
Secondly, we are having and starting to have massive test suites.
Jessica Selvin
So as products uh mature, we don't just add a handful of tests.
Jessica Selvin
We add hundreds maybe thousands and then multiply them by the devices combinations or the OS versions or the browser uh versions.
Jessica Selvin
Right?
Jessica Selvin
Um right.
Jessica Selvin
So adding 800 test cases for example across five devices and different OS versions say poor OS versions just think of the you know permutations and combinations that would involve and running everything um everywhere at every time is no longer realistic.
Jessica Selvin
Uh thirdly, because of these there are shorter feedback loops now, right?
Jessica Selvin
Uh even with a device cloud and plenty of CI uh runners, work piles up and jobs are waiting in cues.
Jessica Selvin
Uh and there's let network latency between your uh code and remote infrastructure.
Jessica Selvin
Uh right and finally because of this there is rising test flakiness, right?
Jessica Selvin
Some of it could be code related uh maybe timing issues or stale locators.
Jessica Selvin
Um but a lot could also be environmental right so there could be network jitters um a shared state between parallel jobs uh devices that aren't isolated well and over parallelization can actually make things worse where everything finishes except maybe a few uh which needs to be retried or rerun uh right so there's a lot of rerun culture and spend time the engineering time uh you know looking into whether a red build is either a real regression I mean real issue or just noise right so all these are some of the realities as QA you would be able to uh you know you'll be able to go ahead and relate to that so with that said let me just go ahead and get into what are the ways that testing is evolving fast right and we see that test automation or scripting is evolving fast with AI but test efficiency is still not yet on par with AI meaning test execution.
Jessica Selvin
So AI has trained us to expect um answers instantly uh and engineering mirrors that with smaller PRs, rapid mergers, right?
Jessica Selvin
Yet our test cycles are taking hours, right?
Jessica Selvin
And even if the tests themselves are quick, uh there could be other latencies that add up uh and make your test cycles much Even in terms of the infrastructure and signal quality gap, right?
Jessica Selvin
So um the real fa failures versus flakiness.
Jessica Selvin
So as we paralyze more, as we're running on more unstable environments uh that might be introducing uh network latency, we see more environment redu uh induced flakiness, right?
Jessica Selvin
Timing issues, resource contention, uh network uh jitter and test data collisions.
Jessica Selvin
Uh right.
Jessica Selvin
So the signal to noise ratio drops precisely when the organization needs faster clearer answers.
Jessica Selvin
Right?
Jessica Selvin
So um we are seeing that in the test automation scripting space but we are not yet there when it comes to the test […]
Jessica Selvin
So um with that let's see how in the overall history of how test automation has evolved and how QA has evolved right um where does intelligent test orchestration fit in right so firstly um talking about manual testing with spreadsheet reporting right early practices that mainly relied on human execution of checklists collation of results on spreadsheets um you know uh so this model was delivering slow inconsistent feedback, limited traceability.
Jessica Selvin
The next was the waterfall methodology that formalized the stages and signoffs.
Jessica Selvin
It introduced more predictability but at the cost of agility.
Jessica Selvin
It was testing still remained a late life cycle activity and any change um in late in the process created substantial rework and schedule risk.
Jessica Selvin
Right?
Jessica Selvin
Then came the automation phase with frameworks like Selenium and APM.
Jessica Selvin
uh right uh which codified repetitive texts, reduced manual efforts.
Jessica Selvin
However, execution still remained largely static, right?
Jessica Selvin
Pull suites were triggered on fixed schedules or broader events and regardless of what had actually changed in the codebase or environment.
Jessica Selvin
So, there was no intelligence in the process.
Jessica Selvin
Uh basically, right, it the your test suite was just scheduled at fixed uh points in time and as and when there was a release, it was re uh it was triggered, right?
Jessica Selvin
Um at the same time agile continuous testing and DevOps comes into play right with pipelines began to run tests on each comet and against every device browser clouds.
Jessica Selvin
So the velocity in terms of the releases increased but the newer constraints from the execution standpoint started coming out right very large suites um right uh contention for shared infrastructure rising flakiness so feedback often arrived too late um because as features are getting shipped the feedback is not coming in as quickly right and finally in the phase five we see the AIdriven test orchestration so this is where we are at and and this is the current inflection point right it's no longer more and more script but better decisions about our execution so intelligent orchestration uh treats testing as a continuous observe decide and act loop right so um be it observing record patterns or recent code changes historical failure patterns uh right and deciding based on that to prioritize what kind of test cases are likely to yield a more meaningful signal right And finally acting on those right execute close to the target infrastructure to reduce latency.
Jessica Selvin
So uh paralyze safely at scale and apply to targeted retries instead of uh everything being retried or your entire regression suite for a particular code change.
Jessica Selvin
Right?
Jessica Selvin
So organizations adopting intelligent orchestration realize three consistent benefits.
Jessica Selvin
So firstly faster and earlier signals uh instead of waiting for hours before you get the feedback uh actionable results are much faster uh in minutes rather than hours right uh higher signal quality so because of the reduced flakiness or the false negatives right reduced noise from the environmental flakiness or network related issues uh right or queuing issues or parallelization issues these are all limited and we are able to have a clearer pass and fail uh semantic and finally improved overall utilization and um in terms of uh even uh efficiency in terms of economics right higher utilization of your existing resources um be it your infrastructure the device browser infrastructure and high utilization of your current capacity and fewer redundant executions right so um AI orchestration in terms of execution is here to stay and it will be one of the crucial components annually lead the way in autonomous testing.
Jessica Selvin
So that's for […]
Jessica Selvin
So let's just take a step back and see a particular workflow uh on how currently uh your scripts why they're taking time to send and receive requests from your test cases, right?
Jessica Selvin
So let's take a typical flow of you are running your um scripts be it from your framework or from your CI/CD.
Jessica Selvin
you triggered it and now you're running it on uh you know a device cloud platform uh and then you're waiting for your results right so um what are the different points uh you know and what is the path that your script would travel and for you to get back your uh feedback right so firstly is the script that you may be executing from your laptop your CI runner right and that talks to the provider hub which is basically the web driver or your APM end point APM selenium endpoint right so what are the challenges here that for every each and every click action right be it if you're familiar with APM or selenium right like find element or click or get text everything sends a tiny HTTP call to the hub and this is passed over the public internet right so which itself introduces unpredictability and latency into your um system so for example imagine that uh there is a latency tax on every call, right?
Jessica Selvin
And even if it's um on a small scale, it might look like a very minute difference like in maybe every call introduces 250 milliseconds of delay per round trip, right?
Jessica Selvin
But if you multiply that by 100 commands or even thousand commands, it will be um 25 seconds or 250 seconds, right?
Jessica Selvin
So that will add and if that's per test case if that is the latency getting added and if you're executing hundreds of thou or thousands of test cases you can imagine the kind of latency uh that will happen right so multiply this by thousands of tests and that's where you will see the hours getting added into your test cycle.
Jessica Selvin
Next is the hub or the browser device node.
Jessica Selvin
Right?
Jessica Selvin
So the hub forwards each command uh to the worker running on the browser or device.
Jessica Selvin
So for a new session the node may boot a browser uh and there could be some predictability predictabilities in terms of you know get uh having the right device availability at that point in time when the request is placed.
Jessica Selvin
So cold starts will be adding seconds to every job and uh right so the queueing will start right and uh this will also be adding to additional time for your test cycle to […]
Jessica Selvin
So and also from the uh artifacts that need to get received back right so be it for our test reporting you'll be collecting maybe the session video or the screenshots or the device logs so all of this page loads API calls all of this uh will add variability again and it can impact the test cycle time so that is to do with the network jitters or dynamic content and as I mentioned right uh where web driver or APM is request heavy design right so because every single request it's very request heavy so every single action step places a separate uh HTTP request so this getting added into every single test case is where your primary latency is coming so that that's where you would see the uh teams would start feeling the pain right because of these systemic challenges uh the distance and the chattiness or the request heaviness that dominate the time so many tiny calls over a long distance will eventually lead to a big overhead, right?
Jessica Selvin
So the browser click may seem fast but the round trip to get that feedback is not and suits often are split by test count and not duration.
Jessica Selvin
So this also can lead to queuing and finally environmental flakiness and reruns.
Jessica Selvin
So uh there could be some uh intermittent reds or false reds teams normalize rerunning to get uh you know uh which inflates the cycle time and hides the real regressions or the real issues right and finally um observability gaps.
Jessica Selvin
So getting the artifacts and logs uh these are fragmented then this will be coming in as a bulk.
Jessica Selvin
So this will also add latency into the system right like a a thing like session video fetching the session video or the screenshots uh right so all these uh are part of every typical in a traditional way of doing test execution these are the different challenges that come into play um which may not be visible directly to you as the QA but this is what is causing those lags and um you know increased test time cycles […]
Jessica Selvin
So to summarize what are the different aspects which are causing this is firstly as I've already highlighted is the network um every single call has to travel back and forth um also screenshots and videos also have to take the same path uh so this leads to the network latency uh test flakiness because of the environment right and network jitters so maybe a network locator has changed or the wait wasn't wait time wasn't long stuff, right?
Jessica Selvin
So these kind of things um you know can introduce flakiness to your tests which need to be rerun again and again.
Jessica Selvin
CI/CD pipelines also can look simple but uh when we're running on on a huger scale, right?
Jessica Selvin
Uh one slow chunk can hold up the entire run, right?
Jessica Selvin
So and then gathering test data.
Jessica Selvin
Um as well as finally just to summarize four things that's are slowing down is that the internet isn't always fast and the tests sometimes fail randomly and pipelines aren't perfectly balanced.
Jessica Selvin
So all these add little delay and together they add up to become a huge delay right and that's why you would be feeling that the feedback is very slow even though when the individual steps may seem quick.
Jessica Selvin
So what are the key factors that are needed for you to have modern AIdriven test execution right that speeds up at scale right let's look at some of the factors in that so firstly aligning with rapid release cycles right so we need feedback while uh you know because as in when the features are developed from the development team the feedback to the development team must be faster right and that means shrinking up the roundtrip delays and getting early pass or fail on the highest risk areas first.
Jessica Selvin
Secondly, handling growing test suites.
Jessica Selvin
So suites grow but so does the device browser matrix.
Jessica Selvin
So running everything and everywhere won't scale.
Jessica Selvin
We need what is required is more of a smart distribution that splits the long tests um and runs um safe parallel tests, right?
Jessica Selvin
That do not slow down your entire test cycle time.
Jessica Selvin
And finally, prioritize tests by risk.
Jessica Selvin
Not every change would be requiring a regression.
Jessica Selvin
Right?
Jessica Selvin
So, we should pick tests that are most likely to catch um a regression based on the code that has changed.
Jessica Selvin
So, uh usage patterns or failure history and that turns it into a signal and not just a simple execution.
Jessica Selvin
And finally, eliminating your test um flakiness and instability.
Jessica Selvin
So random uh reds would drain the reliability of your test results, right?
Jessica Selvin
So we need separate real bugs from the noise and isolate flaky tests.
Jessica Selvin
Apply targeted retries and not simply retry every failed test, right?
Jessica Selvin
Only where the history suggests flakiness and fix the root causes.
Jessica Selvin
And finally uh optimize cost and efficiency.
Jessica Selvin
So you need higher utilization of the grid or the resources of your test infrastructure that you are already having right.
Jessica Selvin
So fewer redundant runs and smarter placement.
Jessica Selvin
So finish faster without inflating the speed.
Jessica Selvin
So putting together what are some of the you know so running sooner getting the PR feedback windows running smarter with risk based selection uh running cleaner with less flaky tests and clearer results and running with better utilization with fewer redundant executions.
Jessica Selvin
So these are some of the key um factors you can say that would play a critical role in ensuring that your test execution is running in an optimal way.
Jessica Selvin
So yeah so to reiterate the point traditional automation isn't enough and with these factors test orchestration with intelligent uh intelligence will be the missing piece right uh so ensuring that you have that in place along with uh intelligence for your automation script development having intelligence and orchestration in your execution as well is the missing piece to ensure higher efficiency.
Jessica Selvin
So with that, introducing T-Cloud's QuantumRun, which is T-cloud's intelligent orchestration engine that executes your existing tests next to the device grid and actively schedules them across real-time capacity, right?
Jessica Selvin
So the two biggest time syncs that we were discussing, right?
Jessica Selvin
Uh the network round trips and the backlog are effectively engineered out of the loop.
Jessica Selvin
So that is what QuantumRun will do for you and it'll power up much faster scaling up your uh test execution anywhere between 10 to 10,000 runs it is able to handle and it executes them next to the grid so that it eliminates those network […]
Jessica Selvin
So what are some of the key capabilities of QuantumRun like as I mentioned 70% faster test execution this is something we have observed directly uh from running uh typically versus running with QuantumRun we see we have observed 70% on average faster test executions so this comes from changing the topology of the execution right not just pushing harder on the grid but QuantumRun packages your suite and runs it next to the devices and browsers.
Jessica Selvin
So it is not taking the requests over the public internet.
Jessica Selvin
Uh this is next to the device browser grid and that removes the van round trips right.
Jessica Selvin
So thousands of fine grained the each and every API call that you see uh is now not traveling over the public internet but over the low latency internal links.
Jessica Selvin
So this results in dramatic time cuts right.
Jessica Selvin
So uh you are able to see as I mentioned 70% faster […]
Jessica Selvin
Next is lightweight uh programming.
Jessica Selvin
So there are no there's no need to make any major changes to your current test scripts.
Jessica Selvin
Right?
Jessica Selvin
So there's no need to rewrite anything.
Jessica Selvin
There is no new test framework as such.
Jessica Selvin
Uh you can keep your APM selenium script uh as well as your CI.
Jessica Selvin
All you need is to zip your project, create a job or a project, set the targets and run.
Jessica Selvin
So QuantumRun will be able to handle the session drivers, placements and the retries as well as giving you the final uh output results, monitoring your sessions through live view.
Jessica Selvin
All of that um are in place, right?
Jessica Selvin
So there is no major changes required to any of your scripts in order to make your QuantumRun work.
Jessica Selvin
Thirdly, faster feedback loops.
Jessica Selvin
Because of the network latency being uh heavily reduced, you'll be able to get those faster feedback loops and you'll get the faster first signal much faster.
Jessica Selvin
Right?
Jessica Selvin
So this will in terms of tracking it from your first pull request to the first result time will be much faster and QuantumRun is designed to pull that number down and keep it stable even as your test suites are growing and also this would reduce your test flakiness and because of targeted and capped retries that isolate the suspected flaky test so they don't clog the main queue and you're having consistent um you know uh results.
Jessica Selvin
So with fewer reruns, clearer uh failures and a build status you can trust, right?
Jessica Selvin
So that is where QuantumRun will help you be it in terms of your faster execution, faster feedback loops and reduce flakiness with minimal change to your code.
Jessica Selvin
Right?
Jessica Selvin
So that is where um we are talking about the key capabilities of QuantumRun.
Jessica Selvin
So let's deep dive into the some of the key tech behind QuantumRun right.
Jessica Selvin
So I'm just going to highlight on four key aspects of QuantumRun.
Jessica Selvin
Uh firstly the proximity execution.
Jessica Selvin
So as I mentioned right uh the first row that you see the first image right in a straight line you have the script hub browser and uh your testing app be it your mobile app or your web app right.
Jessica Selvin
So that is where the latency is introduced because it goes over the public internet.
Jessica Selvin
Whereas here you see that all your uh key components are placed next to your device infrastructure.
Jessica Selvin
Right?
Jessica Selvin
So every tiny yeah every tiny web uh driver or APM command that makes that full round trip over the internet right all that is going to be now happening over a low latency call with the device infrastructure in the same in within the same uh network right so there is no uh network hops and there is no travel over the public internet so that is where you're getting the major uh latency reductions so when you're implementing that to the thousands of test cases, right?
Jessica Selvin
That is where you're going to see those major benefits in the time cycle […]
Jessica Selvin
So yeah, let me just high get deep dive into some of the key aspects of this.
Jessica Selvin
So firstly, the artifacts, right?
Jessica Selvin
Your screenshots, videos, logs are captured beside the run.
Jessica Selvin
So everything is streamed out efficiently because it is not completing over the command traffic, right?
Jessica Selvin
And this leads to latency collap.
Jessica Selvin
Next, the cues are shrinking with faster per command execution, right?
Jessica Selvin
Because of that time reduction and the proximity execution.
Jessica Selvin
The device time is freed quickly.
Jessica Selvin
So the devices are getting freer quickly and the capacity increases, right?
Jessica Selvin
So as in when the test cases are executing faster, the devices are getting freed up for your other test cases uh to run, right?
Jessica Selvin
And this is improving your stability because of the fewer internet hops which means there are fewer timeouts, fewer retries, fewer false positives, right?
Jessica Selvin
And also security and simple simplicity, right?
Jessica Selvin
So there is no fragile tunnels from your CI to devices.
Jessica Selvin
Uh right.
Jessica Selvin
And so the final takeaway is that what proximity execution does is it flips the topology.
Jessica Selvin
It moves your test scripts to the devices where the execution is actually happening, right?
Jessica Selvin
Uh instead of having um communication one by one for each call over the internet.
Jessica Selvin
So this eliminates most of the network latency.
Jessica Selvin
It reduces the queueing and you get faster, cleaner signal without changing or doing anything different to your scripts.
Jessica Selvin
Next is the high performance runtime.
Jessica Selvin
So, QuantumRun is built on the Go language, right?
Jessica Selvin
So, that choice was intentional because it really makes the engine quick, steady, and easy to scale.
Jessica Selvin
So, you're able to do lots of things at once.
Jessica Selvin
Uh, Golang gives us tiny lightweight workers, right?
Jessica Selvin
Where you can run thousands of these at the same time.
Jessica Selvin
It basically it helps in parallelization and finishing together, right?
Jessica Selvin
And secondly, it starts fast.
Jessica Selvin
it compiles into a small single binary right and the containers are light and boots up in seconds.
Jessica Selvin
Um thirdly, smooth uh logs or video streaming, right?
Jessica Selvin
So it uh the input and output very well and it's able to get these heavy data, right?
Jessica Selvin
Like your video session steadily while the test cases are running.
Jessica Selvin
So uh it doesn't jam the pipeline or slow down the test.
Jessica Selvin
It's also stable under load.
Jessica Selvin
So this will really help you scale up.
Jessica Selvin
So even if you're starting small and then you're planning to scale up with your uh test suites, uh it is able to handle that.
Jessica Selvin
And so the bottom line is that it is highly lean and concurrent, right?
Jessica Selvin
So and that makes Turan u to come up fast uh and the performance stays steady even when the system is busy, right?
Jessica Selvin
So the uh building on Golang was an intentional uh decision in order to achieve um better efficiency as well as better um queuing parallelization and stability on the load.
Jessica Selvin
Yeah.
Jessica Selvin
Uh, another key feature of the tech behind uh, QuantumRun is that you're able to have parallel uh, distribution.
Jessica Selvin
So, QuantumRun breaks the sweets uh, into tasks uh, and uses the history to estimate the duration, right?
Jessica Selvin
So, long and short test cases are mixed to create a balance and there is containerized execution.
Jessica Selvin
So each task runs on an ephemeral containerization container with just the tools it needs right so you don't need to worry about um scaling up right and there's smarter placement the scheduleuler looks at the live capacity uh which devices and browsers are free at that point in time and places the task where they will finish faster.
Jessica Selvin
So and also it is able to have failure isolation.
Jessica Selvin
So if a device disconnects or a browser crashes only that single container is retried and it does not impact the entire job right so suspected flaky tests will get a cap retry in isolation so that it doesn't affect the other test runs that are happening.
Jessica Selvin
So that is a you know a quick overview on the parallel distribution.
Jessica Selvin
Next we have intelligent orchestration right.
Jessica Selvin
So um be it look it is able to analyze the code changes uh the historical failure history um right uh test duration live device browser availability right and it's able to decide based uh based on a risk based prioritization right the it's able to score test by the likelihood and impact and it doesn't just simply rerun uh rerun every single failed test right so if a test fails it will pass classify it and see whether it's a suspected flake and then a small isolated retry is done and finally execute near the device grid right so uh it is able to execute the your test cases and it's bringing it to the device grid where your actual devices and browsers are running so this will help to rebalance if the capacity shifts and early failures will be immediately detected so this is what intelligent orch orchestration will bring about uh right instead of running everything it is able to smartly schedule smartly allocate capacity and do the risk based prioritization.
Jessica Selvin
So to summarize uh where are we seeing uh the key differences with the traditional approach versus uh QuantumRun uh right so firstly you're able to see that the remote execution that is added latency is reduced by bringing local like grid near grid execution so the typical uh you know challenge of because I'm executing on cloud devices there is more latency will be eliminated right and that is bringing it up to 70% % faster.
Jessica Selvin
Uh and also the networkdriven flakiness which uh you know um uh isolated workers and devices that are yielding uh consistent results.
Jessica Selvin
uh and instead of running everything the AI selects only um the tests that matter and based on prioritization based on history it is able to make those calls right um faster feedback right so real time uh actionable signals are there uh while uh developers are still having the context so as in when the features are released uh around the same time you're getting the feedback on that instead of there being a delay or a lag in terms of the collaboration between QA and right so finally where QA is becoming a bottleneck QA becomes a accelerator where you're giving faster feedback uh to the developers and your failures and passes are trusted right so traditional execution you are paying latency and a Q tax QuantumRun removes that uh the distance schedules intelligently and it surfaces the you know um signals much faster.
Jessica Selvin
So QA stops blocking and starts accelerating.
Jessica Selvin
So that is where um you would be seeing the differences with QuantumRun coming into the picture.
Jessica Selvin
So with that uh before we move into the demo, I just want to give a highlight about our entire product stack and see where QuantumRun would fit into that.
Jessica Selvin
Right?
Jessica Selvin
So um I'm just going to talk about our entire PC platform architecture here.
Jessica Selvin
So firstly you see that we have our test infrastructure where we are hosting uh real devices and browsers and we also support customized devices and we have autonomous orchestration right this is where our QuantumRun comes into play and it's able to execute your test cases much faster with uh you know uh eliminating the latency prioritization and intelligent parallelization and we also have several AI agents uh that are um present.
Jessica Selvin
So we have our test case generation agent, we have a test script creation agent.
Jessica Selvin
So all these AI agents especially when it comes to script creation, you will be able to accelerate from the test creation side and QuantumRun is able to um accelerate from the test execution uh side, right?
Jessica Selvin
And some other agents to highlight is that we have an observability agent that will give you AIM ML insights into the performance of your application.
Jessica Selvin
We have monitoring agents that will monitor and run 24x7 critical workflows uh in your application.
Jessica Selvin
We have even agent builders and AI systems testing agents as well.
Jessica Selvin
So uh that is just to give you an overall idea of our platform be it functional testing performance experience testing as well as our core device infrastructure of devices and browsers.
Jessica Selvin
So QuantumRun will be a critical component in ensuring that you're able to scale that up and you're able to scale your test cases and test suite executions.
Jessica Selvin
So with that I will move to a quick demo of uh QuantumRun and show you how it works.
Jessica Selvin
So this is uh our platform.
Jessica Selvin
This is our Tcloud platform.
Jessica Selvin
We're a SASbased platform and I'm demonstrating this on our public cloud.
Jessica Selvin
Though we have several modes of deployment uh be it onrem or private cloud options as well.
Jessica Selvin
So as I mentioned we have our real devices and browser infrastructure here where you will be able to see uh that we are hosting actual physical devices across our multiple Pcloudy data centers.
Jessica Selvin
Right?
Jessica Selvin
So we have uh devices across uh different geographies and regions.
Jessica Selvin
So every device that you see here are real physical uh devices right and you can to give you an idea of the range we ensure that we have the latest OS versions um on onto our platform be it Android 16 and 15 and we also support the earlier versions similarly for iOS as well iOS 26 and 18.5 along with the earlier versions of iOS and all the different OEMs like your Apple Google Samsung and so on.
Jessica Selvin
Uh we also have SIM card enabled uh devices to do um test your push notifications, OTP, SMS based scenarios as well.
Jessica Selvin
And as I mentioned we have multiple data centers um across India, the US, Singapore, UAE and few more in the pipeline as well in Canada and the UK.
Jessica Selvin
So that is just to give you an idea about our mobile device infrastructure and similarly we have our infrastructure in uh for Windows and Mac machines for your cross browser testing as well.
Jessica Selvin
So we have Windows and Mac machines and we support all the different um uh versions as well for your browsers.
Jessica Selvin
So for example uh for Google or Firefox, Edge, Safari uh right you will be able to select the versions and go ahead and do your cross browser testing as well.
Jessica Selvin
So that is a bit to give you an idea on our device browser uh infrastructure uh along with some here are some of our agents right like our test creation agent and our performance experience agent um and today I will be deep diving into our QuantumRun.
Jessica Selvin
So let's just have a look at that.
Jessica Selvin
So firstly you can go ahead and create uh a project of your choice right like for example for a new application or new build you can go ahead and create a project give a project name and enter the description and save it right so I'm already having a project here so once you've created the project you can go ahead and create a job under that right so this is what will help containerize as I mentioned in uh during the session right this is what will help containerize your test suite uh separately and ensure that it's not affected by other test execution.
Jessica Selvin
So you can go ahead and create a job.
Jessica Selvin
So let's do that.
Jessica Selvin
So you can go ahead and give a name to your project.
Jessica Selvin
Sorry, a name to your job.
Jessica Selvin
Uh give a um description and select the automation framework that your test cases are written in the language and the framework.
Jessica Selvin
Right?
Jessica Selvin
So once you have selected that you can go ahead and upload the zip file of your project.
Jessica Selvin
So be it uh where wherever your test cases are right getting developed you can just import that sorry export it as a zip file and go ahead and upload it right so there are no major changes as such to be done to your code so I'm just going to go ahead and select a project once selected I'm going ahead and saving this job so you can see that the job has been created successfully now I will come here and go ahead and trigger the run so you can see here that I've create I'm going to create the run and I can go ahead and monitor it over here.
Jessica Selvin
So there's this unique run ID created for that and this will be running containerized right.
Jessica Selvin
So as in when it gets picked up.
Jessica Selvin
So firstly what will happen is there is going to be um it's going to go ahead and download the zip file and ensure that the zip file is uh you know your scripts are running locally near the devices.
Jessica Selvin
So that is what is happening right now to ensure that uh you know you need to uh to ensure that the network latency is minimized right so that is why we're uploading the test cases as a zip file to ensure that the um test cases can be retrieved and placed near the devices so that those network latencies are minimized so that is happening now and the tasks are both in progress.
Jessica Selvin
So here for uh you know the purpose of the demo I'm just showing a couple of test cases.
Jessica Selvin
Of course QuantumRun can scale anywhere between uh 10 to 10,000 test cases and you you know you can go ahead and um sign up for a trial if you want to try it out as well.
Jessica Selvin
So meanwhile as and when the tasks are getting picked up right another thing you can go ahead and do is monitor your active sessions.
Jessica Selvin
So any session uh be it a manual or an automation session uh will get reflected here.
Jessica Selvin
Yeah.
Jessica Selvin
So you can see right there is uh a couple of devices are already picked up and it's ongoing.
Jessica Selvin
So let's open that up and see the live view of the execution.
Jessica Selvin
So the devices so one task is already completed here you can see and there is another one ongoing meanwhile you can also track it from your reporting right so if you don't want to open up the live view use for every single execution uh our live reporting is already there so you don't need to wait for the execution uh to finish right so that is where you'll be seeing um you know faster feed feedbacks uh are happening here where even before the execution is finished, every single run will have uh the PR report or the progressive report uh […]
Jessica Selvin
So you can see here this is my QuantumRun project that we've just now run it on and you'll be able to see the status.
Jessica Selvin
So I can currently see that one of my test cases are still in progress.
Jessica Selvin
Another one that we've just now uh it's been updated about a minute ago has already completed and you can see the execution live here uh sorry the session recording here immediately fetched and retrieved and you can also see the text logs and which is the output of each and every step over here right and what the entire suite will help you see is the status of all your test cases.
Jessica Selvin
So I can see uh the failures.
Jessica Selvin
I can see what have been completed and how many are in progress.
Jessica Selvin
So now even the other test case has also been completed successfully.
Jessica Selvin
So I can see the status of that as well.
Jessica Selvin
So that has been just updated around 25 seconds ago and you can go ahead and see the entire results over here.
Jessica Selvin
Right?
Jessica Selvin
So this is how you'll be able to see instant reporting and imagine scaling this up uh you know as in when you want to scale it up uh to hundreds of test cases you will really be able to see that 70% reduction compared to a live I mean a typical traditional way of executing right so that is where you will as in when you scale up you will be able to see those time differences much more concretely uh in terms of the latency reductions and the time reductions Right.
Jessica Selvin
So also another few features I wanted to highlight is that we have you can go ahead and share these links.
Jessica Selvin
So um you don't need to be a Pcloudy registered user in order to view these reports.
Jessica Selvin
So anyone else in the team uh right any other stakeholders who need to visualize the these reports you can just share it.
Jessica Selvin
They'll be able to view the reports.
Jessica Selvin
You can go ahead and download uh you know the video the logs and everything.
Jessica Selvin
Uh right and also logging bugs right.
Jessica Selvin
So you can directly log it here from uh Pcloudy platform itself into Kira and we have integration with several other bug tracking tools as well.
Jessica Selvin
So you can simply configure it.
Jessica Selvin
It's just a onetime configuration where you uh give your credentials and then you'll be able to log bugs into Jira as well.
Jessica Selvin
So that is how you'll be able to track based with our you know live view uh and reporting and how uploading your test cases and creating a job containerizes your uh you know your test execution and you're able to u monitor it effectively.
Jessica Selvin
Finally coming to the logs here right to our QuantumRun dashboard you'll be able to see um the logs uh these are the maven logs and you'll be able to see uh the status right and you can search anywhere in these logs if you want to if there was in case any errors or failures for example here you can see the success of the test case also marked over here and similarly you'll be able to see the maven logs for every test run right so I hope this gives an understanding and a bit of insight into how Quron works and um I hope this session was insightful for you.
Jessica Selvin
So with that we can go ahead and take any questions that you may have or any other queries.
Jessica Selvin
Um I'm happy to get into the questions.
Dinakar
I've got uh some here.
Dinakar
Let me just share that with you.
Dinakar
So one of the questions that I'm uh I've kind of got something here is how does local first execution reduce latency compared to traditional automation?
Dinakar
I mean you've kind of explained this I think uh a little more depth.
Jessica Selvin
Yeah.
Jessica Selvin
Yeah.
Jessica Selvin
Yeah.
Jessica Selvin
So to put it very simply right uh your typical executions every single step in your APM flow right uh would involve a separate call uh being made uh right so for example if I'm just opening a sample test case you can see that every single um you know every single step that you see here right for example every click every entry right get text right every single step that you see here would involve u a separate call or a HTTP call made over the public internet.
Jessica Selvin
Now what QuantumRun eliminates is that you're able to bring all of those test cases into uh you know the place where our devices are posted right so we are bringing it uh you know we use uh specific storage uh where you are able to download it into our uh data center uh location near the devices.
Jessica Selvin
So this will eliminate uh you know the network latency uh right and every call is not being placed over the public internet but internal uh you know highspeed links instead.
Jessica Selvin
So that is how the latency is eliminated.
Dinakar
I think another followup question uh what makes QuantumRun orchestration different from simply scheduling the tests in Genkins or similar tools
Jessica Selvin
yeah so even when you're having genkins or any other tools right uh what is still missing the missing element is still that latency right uh so that is where uh QuantumRun will really create that impact right because the scripts are coming where the execution is happening.
Jessica Selvin
All right?
Jessica Selvin
So even if you're having uh you know you'reuling it via your CI/CD and things like that um those optimizations will be there.
Jessica Selvin
Yes.
Jessica Selvin
But at the same time the the network latency right those will not get eliminated.
Jessica Selvin
Right.
Jessica Selvin
So that's where QuantumRun uh will be different uh will bring that differentiating value.
Dinakar
Yes.
Dinakar
Sorry, I was speaking on mute.
Dinakar
Uh here's another question.
Dinakar
Uh can we try this on one suite before moving everything?
Jessica Selvin
Yeah.
Jessica Selvin
Yeah.
Jessica Selvin
I mean uh as and when you want to scale up as I said right scaling up is very easy in with QuantumRun and you can go ahead and start uh with a single test suite see how it's working what kind of results you're seeing with that and then you can go ahead and scale it so there's nothing as I mentioned there's no major changes to be done to your code in the first place right so uh it's not that you need to make any major changes as such uh it's simply a matter of um exporting your test cases into a zip file and and then you're good to go.
Dinakar
Okay, there are a few around MCP as well.
Dinakar
Um, okay.
Dinakar
Will it connect to MCP or how does the automation runs when devices are
Jessica Selvin
Right.
Jessica Selvin
How the when devices are chosen?
Jessica Selvin
Okay, I see those as two separate questions with MCP.
Jessica Selvin
Uh yes, we can do that.
Jessica Selvin
We if you want to trigger certain runs from MCP, uh I think that should be possible.
Jessica Selvin
Um I would might need to check a bit on the feasibility, but I don't see any challenges there.
Jessica Selvin
Uh because we already have uh MCP and if you check out our previous webinar, we are able to uh give prompts from your AI, right?
Jessica Selvin
Like cursor or cloud and you're able to see uh you know an execution happening on Pcloudy.
Jessica Selvin
So that should be possible.
Jessica Selvin
Secondly, how the devices are selected.
Jessica Selvin
Uh it's basically you can specify that in the scripts, right?
Jessica Selvin
You can give the specific device name or you can go ahead and just mention the manufacturer like Samsung, Google and it'll pick any available Samsung uh device at that time or you can also specify any um the manufacturer as well as the OS version like maybe Android 15 or Android 16 14 right and it will pick any one of those available uh devices.
Dinakar
Interesting.
Dinakar
Um there's there's there's another one by Pavan.
Dinakar
Um it says, "Yeah, can we use this tool for API testing?"
Jessica Selvin
Uh currently no.
Jessica Selvin
Uh we do not uh have a facility for API testing.
Jessica Selvin
But yeah, it is there in our pipelines and um yeah, we we will be bringing that as well.
Jessica Selvin
But currently we do not support API testing.
Dinakar
Hey, somebody is curious about how is this different from test trigger and lambda test and and so many other competitors that are out there.
Dinakar
Uh how is it different?
Dinakar
Uh is one of the questions that somebody's asking here.
Jessica Selvin
Uh yeah, I mean in some ways you can say it is similar but yeah lot not all lot of cloud platforms are having uh you know orchestration agents.
Jessica Selvin
they support faith to simple traditional uh execution.
Jessica Selvin
Uh right.
Jessica Selvin
So that is where we are actually changing the where the network the network hops itself right.
Jessica Selvin
So other things may be claiming just in terms of other optimizations be it self-healing maintenance or things like that but uh where we are also having self-healing but uh but here where you're reducing the flakiness and the time is by actually eliminating the network latency itself right so that is one of the key uh differentiating features um when it comes to QA
Dinakar
um I mean I've I've got a few more questions questions from uh Sanina itself.
Dinakar
Um but I I think they want to do a deep dive and understand I think we can take those questions offline.
Dinakar
One of the questions was what AI agent is used in Pcloudy.
Dinakar
Uh I know you've kind of mentioned it through when you were talking about the product stack.
Dinakar
Uh but maybe just like to throw some light and maybe we can connect offline as well to
Jessica Selvin
Yeah, sure.
Jessica Selvin
Sure.
Jessica Selvin
Yeah.
Jessica Selvin
So I think you're referring to Qpilot which is our test creation agent.
Jessica Selvin
uh right it runs on enterprise LLMs and uh you'll be able to give an English language description of your uh test scenario and it will generate the script.
Jessica Selvin
So we have a lot of content uh right uh and demo videos we'll be happy to share and we can connect to you offline as well and uh give you a quick demo and if you're interested in a free trial as well we can see how to facilitate that as well.
Dinakar
Great.
Dinakar
Um, I think we'll just do one more question and then we'll I think we can wrap up.
Dinakar
How does how does it pick the right devices OS versions or can I pick a specific model or a browser version?
Jessica Selvin
Yes.
Jessica Selvin
Yes, you can.
Jessica Selvin
You can specify uh up to the specific device that you want it to run on, right?
Jessica Selvin
It's not even in case if you wanted to run on a very specific device, you can.
Jessica Selvin
So Pcloudy all the devices that we have have a unique name uh and we do provide that uh you know you can go ahead and see the unique uh device names in our capability section here.
Jessica Selvin
So you simply need to just select the uh you know um the unique device name and then you can add it to your script and it will go ahead and run it on that particular device.
Jessica Selvin
So you can this is our capability configurator which will generate where you'll be able to see those unique uh device names right.
Jessica Selvin
So if you see here I can come here and I can go ahead and select any particular device of my choice and I will I can go ahead and execute it on that.
Jessica Selvin
Now in case if you don't want it on any specific device I can go ahead and just give specify the manufacturer or uh the OS version.
Jessica Selvin
So that control is definitely with you and you can optimize however you see fit.
Dinakar
Great.
Dinakar
I think uh there are a few more questions that but we'll take them offline since we want to respect everybody's time.
Dinakar
So thank you so much uh thank you Jessica for you know sharing those valuable insights around AI orchestration and I know your presentation has covered a lot of practical strategies that you know people can think through and implement and discuss come back to us so thank you thank you so much uh we also want to thank each one of our participants who have joined in and uh spared time during the day I'm sure some of the strategies that was shared during this webinar has parked a few ideas.
Dinakar
Uh maybe you can take them back, have discussions with your teams and uh get back to us if you have any questions.
Dinakar
We're more than happy to kind of help you out with the journey.
Dinakar
Uh you can reach out to us through our website.
Dinakar
We have a little chat bot.
Dinakar
Uh just click in, enter your details and then we'll get your requests and we can get the conversation going from there.
Dinakar
So thank you once again for being a wonderful patient audience and an interactive one.
Dinakar
Thank you for all those questions.
Dinakar
Uh, good day and happy
Jessica Selvin
Thank you.
