Jeroline
Hello everyone and welcome to yet another exciting webinar from the change leader series break free.
Jeroline
So today we'll be looking at the topic gamechanging trends in test automation top trends to watch out for.
Jeroline
My name is Jeroline and I'll be your host for today.
Jeroline
So before we get started I'd like to go over a few housekeeping rules.
Jeroline
Your line is currently muted but don't worry you can submit your questions during the webinar using the chat option that you find at the bottom of your screen and we'll answer your questions as.
Jeroline
A part of the Q&A sections section that will happen at the end of the webinar and also please note that this webinar is being recorded and uh we'll send out the link of the.
Jeroline
Recording so that you can have a watch later or share it with your friends and colleagues.
Jeroline
Next slide please. So for those who may not be familiar with Pcloudy, we are a cloud-based testing platform offering 5,000 plus real devices and browser combinations for mobile and web app testing.
Jeroline
Key features of Pcloudy include continuous testing, DevOps, advanced automation, AI based visual testing, robotic process automation, real-time test analytics, and much more.
Jeroline
With integration with popular testing frameworks and tools and round-the-clock premium support, Pcloudy is trusted by 300K plus registered users.
Jeroline
Right from you know small businesses to large enterprises.
Jeroline
Next slide please. Now let's see about break free.
Jeroline
So break free was an idea that birthe out of the need for change in testing domain.
Jeroline
So we've seen the testing industry evolve and grow over the decades as a few leaders took the risk to break free from traditional approaches towards testing and come up with a you know a.
Jeroline
Lot of new ideas and practices and we believe that this change can be accelerated at a much faster pace and.
Jeroline
So we took the liberty of identifying and inviting a few change leaders who are breaking the mold and embracing change to see the QA industry evolve.
Jeroline
And that's what breakthrough is all about.
Jeroline
Our hope is that these sessions would spark ideas in each one of you to be the change in the QA industry for the industry to grow into greater leaps and bounds.
Jeroline
Uh so with that let me welcome the change leader for today Lysa Rakkesh.
Jeroline
Next slide please. Lysa brings her over 17 years of experience in IT consulting solutioning and service delivery of complex engagements.
Jeroline
She has worked with clients across multiple geographies helping them identify their strengths, areas of improvement and designing transformation road mapaps to achieve their organizational goals.
Jeroline
Lysa is also a certified Canvan Saf and SE uh six sigma professional and has worked as an enterprise agile coach to implement agile projects and facilitate transitions from waterfall to agile.
Jeroline
Currently, Lisha is a senior director at Capgeemini where she's responsible for managing and growing um strategic accounts within the financial services business unit and she's.
Jeroline
Also leading the automation and estate capability unit focusing on building the automation competency and future ready test engineers and estates within the organization.
Jeroline
So without any further ado, let me hand it over to the speaker for today, Lisha.
Lisha Rakesh
Thank you Jeroline. Uh that was a very detailed and nice uh you know introduction.
Lisha Rakesh
Uh it felt good to hear good things about myself and thanks to the Pcloudy team for having me here.
Lisha Rakesh
Right. So as Jeroline was talking about the industry is changing right we keep on hearing about the digital transformation initiative.
Lisha Rakesh
Anybody that you speak to within the industry, within any of the organizations or verticles, they will talk about the need to be faster, the need to be going, you know, with cheaper cost or.
Lisha Rakesh
Reducing the operational cost and at the same point of time being very very efficient in the work that we do.
Lisha Rakesh
Now when we talk about the speed at which we are moving, obviously to meet that we need to ensure that the testing is.
Lisha Rakesh
Also being done at the same level, right?
Lisha Rakesh
Because it wouldn't make sense for us to deliver uh softwares or applications in to the production environment with a lot of errors or bugs.
Lisha Rakesh
Right? So, automatically the testing, the validation, the whole quality engineering aspect comes into picture and test automation is one of the areas which helps us or supports us in delivering that to the industry.
Lisha Rakesh
And that's why we thought that we should talk about what are the trends, what are the patterns in the industry and why automation or test automation is.
Lisha Rakesh
So so important. Right?
Lisha Rakesh
So in today's seminar uh we will you know have uh these uh topics.
Lisha Rakesh
One of the reasons why we are talking about automation or test automation is because there is a huge business opportunity around this whole topic.
Lisha Rakesh
Right? And uh I wanted to let you know a little bit about what is the market potential so that any of you who are looking forward to make uh a career in the test.
Lisha Rakesh
Automation industry or who all are already a part of this industry can understand the length and breadth and also the kind of opportunities that you will have if you stick to this particular field.
Lisha Rakesh
The other area while we talk about the future we should also look at how the evolution of automation testing has been in the past.
Lisha Rakesh
Right how it has evolved if you look at you know 20 years back and today how has the evolution been what are the changes that have come in and these evolutions will be uh.
Lisha Rakesh
In terms of processes tools technologies everything so we'll talk a little bit about that and then we'll come to why uh automation is seen as an enabler for better future and we need to.
Lisha Rakesh
Have automation as it's a given now right and we need to have it as a very very critical critical component when we are planning our test strategies and what are the key trends that.
Lisha Rakesh
You should be looking for what are the key things that you should be aware of as somebody or as a practitioner in the industry for automation and the last is how do you prepare.
Lisha Rakesh
Yourself for your automation journey right what are the key things that you should keep in mind what is the mindset and what is the uh you know what are the challenges and what are.
Lisha Rakesh
The key things that you should always incorporate when you are embarking on this journey.
Lisha Rakesh
Okay. Now um one of the things that I wanted to bring across to you right this is coming in from the IDG's uh annual state of the CIO uh report for 2022.
Lisha Rakesh
There are three key areas when we talk about the digital transformation or transformation in whole right there are three key areas which we.
Lisha Rakesh
Look at okay which are called the three playing fields.
Lisha Rakesh
One is the need to go digital, the need to ensure that you are transforming because if you look at the current market or if you.
Lisha Rakesh
Look at the current industry, the user base is absolutely different from what it was yesterday and tomorrow's user base is going to be absolutely different than what it is today.
Lisha Rakesh
Right? So if you have to ensure that you're walking at the same speed or running at the same speed at which the industry is moving, you need to ensure that you are transforming continuously.
Lisha Rakesh
Okay, the next thing is around the customer first or the user experience first mentality or mindset.
Lisha Rakesh
Everybody talks about customer engagement.
Lisha Rakesh
It's very very important.
Lisha Rakesh
We have got applications which are global in nature.
Lisha Rakesh
We have got people who are uh so connected right and anything that goes wrong uh gets a lot of attraction and a lot of attention from the whole globe.
Lisha Rakesh
So it is absolutely imperative for you to ensure that you are delivering the softwares or the applications to your customers which are friction which are giving them very very uh customized and frictionless experience.
Lisha Rakesh
Right so something which is very smooth something which is very easy to navigate and also understand that the community that we work with these are communities wherein you have got children who are accessing.
Lisha Rakesh
Your applications you've got adults who are very busy, right?
Lisha Rakesh
Who have very less time to navigate through the applications etc.
Lisha Rakesh
And then you've got the elder population also who might not be very digital savvy.
Lisha Rakesh
So when you think about it, you need to ensure that anything that you're building is having that customer first mentality and you being able to satisfy the needs of your customer.
Lisha Rakesh
The next one is definitely around the operational efficiency.
Lisha Rakesh
How do you reduce the cost while you still get the right things rolled out?
Lisha Rakesh
How are you being able to reduce the cost of development?
Lisha Rakesh
How are you being a able to reduce the cost of testing and still being able to meet the objectives?
Lisha Rakesh
Now you might have heard about certain things in uh you know in the past I think last three five to five years we've been talking about data we've been talking about cloud.
Lisha Rakesh
Right everything is moving to cloud uh we've got our customers who want to move to cloud if they've not yet moved and.
Lisha Rakesh
Then there is this whole uh data concept right wherein we dealing with such huge amount of data and the there are insights that are coming in which can help you in ensuring that you're.
Lisha Rakesh
Taking right decisions for your business.
Lisha Rakesh
But if you look at both these areas, you would still see that to encompass all of this right and to deliver the objectives or the playing fields that we call you still need to.
Lisha Rakesh
Have a very strong pillar of automation which you can leverage in order to satisfy your end customer or your business.
Lisha Rakesh
Okay. So I wanted to bring this across to you that no matter what you're talking about if you are given raw data uh high volume of data and you are asked to go ahead.
Lisha Rakesh
And ensure that uh you know you make decisions based on that data you would not be able to do that unless and until you use some automated mechanism to get the.
Lisha Rakesh
Right kind of insights from there.
Lisha Rakesh
Similarly if you're talking about cloud the whole idea about cloud is going faster.
Lisha Rakesh
Now if you take a whole lot of time to ensure that you're validating the uh releases or you're validating the applications before you moved into cloud, it's again going to be a very big.
Lisha Rakesh
Challenge. So that's what I want all of you to understand that automation remains the key no matter what is the underlying technology that you're using.
Lisha Rakesh
Right? With that we look at what is the market opportunity because end of the day it's all about the bread and butter that uh we uh earn.
Lisha Rakesh
Right? So if you look at the 2022 data, automation testing market was evaluated to be at 24.7 USD billion dollar.
Lisha Rakesh
Right uh and uh if you look at what is the projection looking at the kind of um business or kind of growth or the pattern that we have seen uh there is a clear.
Lisha Rakesh
Projection of 52.7 U billion USD by 2027.
Lisha Rakesh
So just imag imagine the kind of growth right keeping in mind the industry trends right now it's a you know growth rate of 16.4%.
Lisha Rakesh
Which makes it a very very lucrative industry to stick on to right.
Lisha Rakesh
Uh if you look at the market segment analysis you would see that you know the highest revenue generation is done through the North American um uh industry and that's the highest revenue contributing geography.
Lisha Rakesh
But the fastest uh pace at which one of the geographies is growing is the APAC right you see a lot of traction and a lot of move towards uh projects which are you know.
Lisha Rakesh
Getting into automation testing in the APAC market.
Lisha Rakesh
Okay. Um so and this is something that is projected to happen over the next 3 years at least wherein you would see still the revenue would probably be coming from the North American um.
Lisha Rakesh
Uh region because of the high volume and uh the size of the industry but the traction is still seen to be more in the APAC region.
Lisha Rakesh
Now let's talk about like I said while we talk about the future we also need to be uh aware of how this industry has evolved.
Lisha Rakesh
So when I started it it was like 18 17 18 years back.
Lisha Rakesh
Uh at that point of time I think automation was something which was gaining momentum.
Lisha Rakesh
It was very niche at uh that point of time and very specialized and anybody who could code and who had the testing knowledge was moved into automation testing and that is what is the.
Lisha Rakesh
Evolution that you see here right uh if you look at how the whole automation part start because at one point of time if I.
Lisha Rakesh
Look at 20 years back we didn't know automation testing was definitely very important it has stayed uh to be very relevant and very important over the years but it was all functional testing manual.
Lisha Rakesh
Testing wherein you would have testers sitting down right understanding the requirements working with the BAS working with the business creating the manual test cases executing them manually all of that.
Lisha Rakesh
Then when the automation testing tools came into the market uh people started getting interested for that right and they started looking at uh trying out automation uh the tools were probably not very formally.
Lisha Rakesh
There in the market uh there were utilities that were being used to uh ensure that you could bring in some efficiencies into your work,.
Lisha Rakesh
Right? So there was no end to-end automation but you were doing some of the things maybe you know file comparison or something like that.
Lisha Rakesh
So it was very unstructured at one point of time when it started and there was very minimal automation because people were not very clear about how automation can be used uh what are the.
Lisha Rakesh
Benefits that will come in and uh how accurate it would be.
Lisha Rakesh
Right? With that it moved to a need-based uh automation which meant that certain kind of programs would need automation.
Lisha Rakesh
Because you wanted to run faster you wanted to reduce the cost.
Lisha Rakesh
Okay. I still remember when I started testing at that point of time we had a regression cycle which ran for 3 months.
Lisha Rakesh
Okay. And we had close to about 3,000 test cases which had to be executed uh every 3 months.
Lisha Rakesh
And we had a good big-sized team because there was a problem with the data.
Lisha Rakesh
The environment would not be available.
Lisha Rakesh
You would probably use the same environment with you know many other teams.
Lisha Rakesh
So there was a problem with the environment availability.
Lisha Rakesh
Uh uh there were always issues with you know the functionality understanding etc.
Lisha Rakesh
Right? So when that happened it became very very difficult.
Lisha Rakesh
It was uh project need-based automation wherein you would probably focus only on the regression part of it.
Lisha Rakesh
Uh or you would look at automation if you ended up having a problem in the production environment.
Lisha Rakesh
So something went wrong and then you would get a uh you know mail uh from your senior your senior leadership saying that you need to fix this.
Lisha Rakesh
Okay. So ensure that it is included as part of the automation testing.
Lisha Rakesh
While you did automation you would still do manual testing to ensure that the automation is working fine.
Lisha Rakesh
Then it became more proactive and people started seeing the value right when the three-month uh cycle became reduced to a week cycle people started seeing the value that was being derived.
Lisha Rakesh
Right it became more structured uh the focus shifted more from test cases to business processes and when uh you know you had the API testing coming into picture you had business flows or end-to-end.
Lisha Rakesh
Business flows coming into picture people started talking about downstream upstream automation all of that now it is more industrialized where which means that people are thinking about automation.
Lisha Rakesh
When you think about a new project, you don't think about manual testing at all.
Lisha Rakesh
The first thing that comes to your mind is how can you achieve decent amount of automation coverage in your engagement,.
Lisha Rakesh
Right? The skills also have changed.
Lisha Rakesh
Earlier the skills were hybrid testers, manual testing plus automation testing, little bit of coding knowledge is all okay.
Lisha Rakesh
Now nobody talks about automation testers.
Lisha Rakesh
Everybody's talking about fullstack testers.
Lisha Rakesh
People are talking about ESDES, right?
Lisha Rakesh
And then there is this whole idea around the CI/CD pipeline, your automation testing getting embedded into the pipeline and you being able to churn out test cases, getting things tested and.
Lisha Rakesh
Then pushing them into the production environment.
Lisha Rakesh
And of course the baby steps are taken towards the intelligent automation which means that you are talking about how your automation scripts are executed.
Lisha Rakesh
You know one-touch execution maybe you get to know that this is something that is changed.
Lisha Rakesh
There is a release that needs to be happened.
Lisha Rakesh
You click a button the automation tests uh execute.
Lisha Rakesh
Uh if there is a failure they selfheal.
Lisha Rakesh
They understand what went wrong and then they go ahead and selfheal themselves.
Lisha Rakesh
There's a lot of analytics coming into picture.
Lisha Rakesh
You're looking at how the production systems are behaving and then pushing it back to your automation uh teams or testing teams.
Lisha Rakesh
So that they can understand what needs to be tested and they can optimize their uh whole efforts.
Lisha Rakesh
Right? So all of this is how the industry has evolved.
Lisha Rakesh
Again, if you look at the last 20 years bracket, you would see in the last 15 years, it was still gradual,.
Lisha Rakesh
Right? And all of a sudden, there has been a very steep jump in the kind of capabilities that have come in,.
Lisha Rakesh
Right? Uh in the automation space, it's it's intelligent, it's very smart.
Lisha Rakesh
Uh and uh there is a lot of expectation that is coming in uh in terms of the kind of people that we need to run uh the automation testing that we have.
Lisha Rakesh
Right now being embedded into the projects.
Lisha Rakesh
Now what does the future look like?
Lisha Rakesh
How does the future look like and how do we prepare ourselves for that is what we will talk next.
Lisha Rakesh
So one of the key things that we've seen people moving towards is the life cycle automation.
Lisha Rakesh
Again if you remember older days right uh people who would have been there in the industry for 10 years above the focus was always on the execution part.
Lisha Rakesh
Right we talk about the execution of testing taking very long time and uh how do you ensure that you're being able to cut down on the time.
Lisha Rakesh
So that your testing team doesn't take months to validate what you have done.
Lisha Rakesh
Okay. Uh but there is always there's always been a challenge with that because when you look at the time crunching.
Lisha Rakesh
Right no matter how many people you put in, no matter how much effort you put in, you would see that you would not be able to still reduce that after a significant period of.
Lisha Rakesh
Time. Because when you talk about execution, it's not just about the test script.
Lisha Rakesh
It's about the whole ecosystem.
Lisha Rakesh
Which means that you have to look at the test data, you have to look at the test environment, you have to take care of your team.
Lisha Rakesh
Because everybody's uh you know planning leaves or something like that and you need to have the right backup.
Lisha Rakesh
There is a knowledge transition or all that people element comes into picture.
Lisha Rakesh
So when you look at the automation only in terms of the test execution, you would never be successful.
Lisha Rakesh
So what you have to do is you have to stretch on both side.
Lisha Rakesh
You have to look on the you know phases before the test execution.
Lisha Rakesh
You have to look at the phases after the test execution.
Lisha Rakesh
One of the challenges that my test managers always come and tell me is that uh Lisha you're spending too much of time creating reports.
Lisha Rakesh
Because we have huge teams.
Lisha Rakesh
The teams are of 200 testers 250 testers and uh everybody's updating it uh and we don't really have a very good mechanism to showcase the.
Lisha Rakesh
Right reports. Why don't we think that we would be able to do the reporting in an automated manner?
Lisha Rakesh
Okay, I still see organizations struggling with the kind of reporting that is being produced unless and until they have a very uh nice sophisticated mature reporting solution.
Lisha Rakesh
Everybody would struggle with that.
Lisha Rakesh
We when we talk to customers two three things always come back.
Lisha Rakesh
One is the test data.
Lisha Rakesh
People always say and the testing teams always crib about the test data and the quality of the test data.
Lisha Rakesh
Again you have got a lot of uh issues around the privacy and you need to ensure that it is properly masked and all of that.
Lisha Rakesh
So test data always remains a problem.
Lisha Rakesh
Why don't we look at automating this whole test data right?
Lisha Rakesh
Again uh on the environments why can't the environments be spinned up?
Lisha Rakesh
Do we really look at test automation in a siloed manner or we look at it through the whole life cycle?
Lisha Rakesh
Just imagine a scenario wherein you would have a BA who would create a model based uh or a model for you which talks about what is the business requirement.
Lisha Rakesh
Your manual test cases get churned up based on the flow that they have given to you automatically based on uh the inputs that are there in the manual test cases.
Lisha Rakesh
Your uh framework would look at it or your people would look at what are the key things that you need to do and you will just plug and play uh what are the functionalities.
Lisha Rakesh
And then you have the data getting churned up automatically you have the environment getting spinned up automatically you get the execution done with the click of a button and you have automat automated reports.
Lisha Rakesh
Coming in just imagine the scenario right so that's why I think we as test managers or even people who.
Lisha Rakesh
Actually doing the testing because you are the ones who are spending so much of time doing all of these mundane uh activities.
Lisha Rakesh
So look at how you can spread or extend automation across and not just have it focused on test execution.
Lisha Rakesh
I think most of us understand it but the implementation is always a challenge because we keep on hearing the same problem when we go to very mature customers.
Lisha Rakesh
Also who are very mature in the technical space technology space but still you hear about the same old problems coming up.
Lisha Rakesh
So there is you know we we need to look at automation as a holistic solution not focusing only on a certain phase.
Lisha Rakesh
The other part is around the low code no code platforms right uh and this is something I'm sure all of you if you have been a part of the automation industry would have heard.
Lisha Rakesh
U uh there is like I was saying earlier when I started uh the talk that anybody who could code.
Lisha Rakesh
Right 15 years back anybody who could code was treated as an autom and who know knew testing right could become an automation tester that is all that you needed you know a programming language.
Lisha Rakesh
And you would learn a tool and then you would get into automation testing.
Lisha Rakesh
That's the easiest career progression that manual testers had.
Lisha Rakesh
Now there is a concept of citizen development.
Lisha Rakesh
Uh I'm not sure how many of you would know about that.
Lisha Rakesh
But the whole idea is that everybody can automate.
Lisha Rakesh
It is not just restricted as a very specialized skill only for people who know how to code.
Lisha Rakesh
Okay. And the whole reason about for this is because you have got people who are earlier or who were earlier working only in certain areas.
Lisha Rakesh
For example, a BA a BA would come in during the requirement gathering phase and then would come at the end when you are doing the demos of whatever you have tested or whatever you.
Lisha Rakesh
Have developed to validate that.
Lisha Rakesh
Why not leverage these VA across the cycle?
Lisha Rakesh
But how do you do that?
Lisha Rakesh
Because you cannot ask somebody to learn coding and you know create that logic and start creating automated scripts overnight.
Lisha Rakesh
Right it's not possible and practically you know not feasible.
Lisha Rakesh
So what we do is we look at low code no code platforms which gives you the ability uh to have non-technical or people having non-coding background.
Lisha Rakesh
Also be able to code to a certain extent execute the scripts and also maintain your framework right or maintain those scripts going forward.
Lisha Rakesh
Because there is hardly any coding required and there are very sophisticated tools in the market or platforms in the market which will give you this ability.
Lisha Rakesh
So if you have uh and this has always been a challenge because what always happened is that earlier when you would do automation for any of I'm talking about the service- based industry.
Lisha Rakesh
Right service- based organization when you would do uh automation uh uh testing for a particular customer it's basically like a marriage you you don't get out of there.
Lisha Rakesh
Because there is so much of dependency on the code that you have developed for them that it's very difficult for them to come out of there.
Lisha Rakesh
So that's why this whole concept and the maintenance effort comes into picture because you know I cannot understand the code somebody else has written.
Lisha Rakesh
Because probably that code is not uh very well um you know drafted or you don't have right comments and everything.
Lisha Rakesh
So I'm not able to understand and hence the maintenance effort goes uh you know takes takes the whole ROI out of the picture.
Lisha Rakesh
So this low code no code platforms enable the complete team to come together and support the test automation uh activity.
Lisha Rakesh
You can have people who know coding they can definitely run it but you can also have a wider user base who can use these tools and create test cases execute and keep them maintained.
Lisha Rakesh
Over a period of time.
Lisha Rakesh
Um then we have the digital experience testing as I was telling that the user base or the consumer base of our applications have is very very varied.
Lisha Rakesh
Right what I would expect from a particular application is very different to how my son my 9-year-old son would perceive it and it will be very different from what my uh you know 65year-old.
Lisha Rakesh
Father would perceive it right so when you look at this kind of a varied user base in terms of the work that they are doing in terms of uh the way they are interacting.
Lisha Rakesh
With your applications the way in which uh you know the kind of devices that they are using right uh if you.
Lisha Rakesh
Look at all of these permutation and combination the key thing becomes or everything the core becomes the experience that you're giving to the customer I would love to go back to an application which.
Lisha Rakesh
Gives me a very seamless experience of using the application but if it is highly difficult for me to navigate if I'm not able to understand or it it just breaks down anytime I try.
Lisha Rakesh
And retrieve some data out of it it's not a very nice experience for me so I would not want to return that and everybody wants that user loyalty everybody wants people to come back.
Lisha Rakesh
To their uh applications um and so that because that's good for the business so when we are talking about this we need to ensure that we keep user experience in the core of our.
Lisha Rakesh
Design principles which means that we look at how well uh the people would perceive our u applications how easy it is going to be for my application to be used by people this this.
Lisha Rakesh
You know in the past one uh two years I've always heard about two key strategic concepts or two key strategic topics in any of the discussions that we have had with our customers or.
Lisha Rakesh
At the board level or whatever.
Lisha Rakesh
But the the the two uh concepts are around the sustainability and the other one is around the inclusion.
Lisha Rakesh
Right? When we talk about inclusion, we also need to ensure that the experience that we are thinking of providing to our customer is including people who would probably have problem in accessing our application.
Lisha Rakesh
So you have to think about the accessibility testing the whole idea around uh security performance.
Lisha Rakesh
Uh again I'm taking you back to the times when I was an automation tester.
Lisha Rakesh
We only looked at functional testing.
Lisha Rakesh
We only understood what are the functional validations that we have to embed in our automation tests.
Lisha Rakesh
Right? We never bothered about the performance.
Lisha Rakesh
Uh security was you know never even considered user experience, accessibility maybe for some applications you know which uh required in financial industry if you required specific applications there were some applications where inaccessible but.
Lisha Rakesh
That was again a very niche area but all of this performance security uh experience and all of that was never compatibility testing they were never a part of what we did earlier.
Lisha Rakesh
It was only functional functional function but I think that if somebody's thinking like that it's absolutely not going to work.
Lisha Rakesh
Right now because you have to ensure that you're giving the same experience to somebody who's uh using uh a MacBook uh or who's using uh an an Android device uh who's using it on.
Lisha Rakesh
Chrome who's using it on edge you need to ensure that everything works well and for that and if you have to imagine that you are uh sharing or uh creating a no use case.
Lisha Rakesh
And you're implementing a news user story and uh you have to get this tested on 15 different platforms.
Lisha Rakesh
Uh so if you would do it only using manual testers or you do it on automation for some of the um uh areas and not on the others, you've already lost the game.
Lisha Rakesh
So the digital experience testing is going to be key.
Lisha Rakesh
Uh it is already key and it is going to be extremely extremely critical going further.
Lisha Rakesh
So we all need to ensure that we are keeping this in mind when we are building our strategy when we are building our frameworks when we are looking at what kind of testing tools.
Lisha Rakesh
And what kind of people that we want in the team to run the projects.
Lisha Rakesh
The next one I think is a favorite topic.
Lisha Rakesh
Uh everybody talks about this whenever we talk about automation trends is about the a IML enabled or driven automation.
Lisha Rakesh
I'll again uh share a use case with you.
Lisha Rakesh
Uh there was one of the projects that we were running.
Lisha Rakesh
It's uh like about few years back we started a project and uh we were about 80% 85% in the automation coverage.
Lisha Rakesh
Okay. because that was what the uh design of the s so was that we had to ensure that we have to give them a certain percentage of automation uh coverage.
Lisha Rakesh
So we were doing that right and what always ended up happening is that when you would go into production you would end up having certain issues in certain areas.
Lisha Rakesh
Right. So um and of course there are SLAs and KPIs and all of that get impacted whenever you have production incidents.
Lisha Rakesh
So there was this problem statement that was given to us that you have to make your you know your testing more effective.
Lisha Rakesh
You have to ensure that there are no production defect leakages.
Lisha Rakesh
So what we did was we took up the whole extract of the incidents right and I'm sure most of you would have done that you would take up all the production incident extract we.
Lisha Rakesh
Looked at all of it and then we said okay uh this is what we will uh do we understand that there are certain areas which are problematic we end up getting more incidents there.
Lisha Rakesh
So we will ensure that we test them thoroughly right so we will ensure that we'll have more people and we'll run more tests and do double or triple times.
Lisha Rakesh
So that we are not ending up having issues there and we did that and still we had issues in the production environment.
Lisha Rakesh
Now why was that?
Lisha Rakesh
We realized that there was a code change that went in which impacted something else.
Lisha Rakesh
Okay. So you know all the effort that we built in and this was all done manually by the way.
Lisha Rakesh
There was no automation used in the production incident analysis and everything but all the effort that we put in it was not really useful.
Lisha Rakesh
Because the efficiency again was not there.
Lisha Rakesh
We still ended up having the same kind of a problem.
Lisha Rakesh
So then what happened was we started looking at how is the user behavior in production.
Lisha Rakesh
What are the areas which the users use more?
Lisha Rakesh
Which are the areas that get impacted if a certain change is coming in?
Lisha Rakesh
Right? And uh what would be u you know the kind of traffic is there a pattern that we see in terms of the traffic which becomes higher and.
Lisha Rakesh
Then people start facing a challenge and all of that.
Lisha Rakesh
So then this whole um uh you know view was put in and we looked at it from a different lens altogether and.
Lisha Rakesh
Then we were able to make changes in certain ways wherein we would get to know.
Lisha Rakesh
The first thing that we started doing was look at the production data, get the insights, we would look at what are the changes, we would.
Lisha Rakesh
Then tie it back or marry that and then we would plan our automation efforts or testing efforts for that.
Lisha Rakesh
Right? So that similarly I think one of the use cases which is very very popular uh in our customer base is around the duplication of test cases.
Lisha Rakesh
I'm I'm sure all of you who are working on automation would have encountered projects wherein you have thousands and thousands of test cases you run during the regression.
Lisha Rakesh
And these test cases are test cases that have been created over years, right?
Lisha Rakesh
Because you've worked on these test cases.
Lisha Rakesh
You've uh you know somebody else has also worked in on it.
Lisha Rakesh
And when you have a a test case uh suite of 1,500 test cases, you would not go back and.
Lisha Rakesh
Look at each and every test case and then create your test case.
Lisha Rakesh
Right? So over a period of time, what happens is that gradually when you start building it like this, you would end up having duplicate test cases.
Lisha Rakesh
And uh there is always a need to do this uh manually uh or revalidate these test cases.
Lisha Rakesh
Uh and most of the times it's manual effort that is put into picture because people will look at it and.
Lisha Rakesh
Then you'll havememes who will come in they will see and they will say okay this is the duplicate you can get it out you don't need to run it or this is an obsolete.
Lisha Rakesh
Test cases etc we used to do a similar kind of a thing uh every six months a good amount of effort would be spent we will sit with the business team we will understand.
Lisha Rakesh
What is good what is bad what is duplicate and then do a whole revalidation exercise for our uh automation uh uh suite.
Lisha Rakesh
But then this whole AI concept came in and our COE team brought in uh the thought process around looking at the duplicates through the NLP um uh you know algorithms and.
Lisha Rakesh
Then giving us and it was so easy for us because the only thing that we had to give them was the extract and they came back to us and they said.
Lisha Rakesh
Okay these are the 100 test cases which are duplicates in some way or the other and they also gave us what they thought was the duplicate uh you know in.
Lisha Rakesh
So which are the duplicate test cases and what are the sub you know what are the areas wherein there is a duplication and that became you know the whole revalidation exercise stopped.
Lisha Rakesh
So our testers were not doing that with the uh business teams.
Lisha Rakesh
They were just using it and it was so easy because you would get the the whole exercise that you were doing over weeks that got reduced to a day or two and you would.
Lisha Rakesh
Get the information we would just send it back to the BAS and we would say that these are the changes that we are bringing in in the.
Lisha Rakesh
Next release. These are the test cases that are being filtered out and it just ran seamlessly.
Lisha Rakesh
So there is always a way to make your testing and your efforts towards testing automation or manual to be reduced.
Lisha Rakesh
Right or to make it more optimized and all of these uh you know AI concepts uh if you can really put across your problem clearly it is very easy for you to understand if.
Lisha Rakesh
There is a solution and there would definitely be a solution right especially with all these generative AI uh tools coming into picture I'm sure none of the problems that we talk about are uh.
Lisha Rakesh
Without a solution. So if you the only idea is if you can really state the problem statement clearly to somebody who understands how this works it would become easier.
Lisha Rakesh
So as practitioners we should look at areas which can be optimized.
Lisha Rakesh
We should look at areas uh wherein we can bring in uh the AI concept to make the working make the efforts more uh fruitful rather than doing the same kind of work and.
Lisha Rakesh
Then the last one is all around the test automation platforms.
Lisha Rakesh
Now when we talk about test automation platforms we all know right when we talk about flows we're not talking only about the web automation we talked about uh that in the evolution.
Lisha Rakesh
Also that it was very UI focused at one point of time but when we talk about end to end testing we're talking about web we're talking about um database we're talking about API we're.
Lisha Rakesh
Talking about uh uh mainframes in some cases we're talking about mobile applications so all of this now if you're working with three different kinds of frameworks and.
Lisha Rakesh
Then uh you know you're maintaining all of them separately and then you have assets for mobile testing you have access for web testing separate uh anybody who's coming in doesn't know where to.
Lisha Rakesh
Look for because you have got different different projects and everybody is working in their own uh spaces it's very chaotic.
Lisha Rakesh
It might not be chaotic for smaller organization but for big organizations especially you know the ones which are dealing with a lot of data volume or a lot of different projects that are running.
Lisha Rakesh
Simultaneously it is very very cumbersome and you end up spending a lot of effort or there is a lot of rework effort that happens.
Lisha Rakesh
Because you really don't know that what has already been done and there is no place wherein you can easily find that out.
Lisha Rakesh
So the whole idea behind these test automation platforms and again there are certain very very good uh uh you know organizations and vendors who have developed these test automation platforms which you can.
Lisha Rakesh
Look at uh you know you just need to do a bit of a research and you can look at the kind of working and the kind of efficiencies that they would provide.
Lisha Rakesh
You can create end-to-end test cases spanning across these different technologies.
Lisha Rakesh
Uh you can use uh visual uh mechanisms to do the validation of the test cases so that it is mimicking your exact user behavior.
Lisha Rakesh
The scripts are uh very modular in nature.
Lisha Rakesh
Uh you can have them created using normal English language because these platforms typically are low code uh or scriptless platforms.
Lisha Rakesh
So you don't really need to worry about coding etc.
Lisha Rakesh
Uh there is a lot of flexibility also that you get because you are able to churn out scripts quickly.
Lisha Rakesh
You're able to understand what are the scripts or what are the assets that are already available to you.
Lisha Rakesh
You get the data in one place.
Lisha Rakesh
You don't have to run to different people and different portals to get the data.
Lisha Rakesh
And the biggest um uh you know advantage that I see when you use a platform is the the feeling of community.
Lisha Rakesh
Uh because uh no matter how good we are uh from our side we would always need some support some help some uh place wherein you can brainstorm and the test automation platforms provide you.
Lisha Rakesh
A very very good way of doing that right with that I'll uh quickly talk about what we need to do.
Lisha Rakesh
Right in order to gear up uh and ensure that we are all ready for this so there are two aspects that I want to bring across one if you are a leader.
Lisha Rakesh
Okay, if you are somebody who's running a practice or if you're somebody who's running uh uh an automation portfolio, you are delivering it for a customer or whatever,.
Lisha Rakesh
Right? So, if you are working in that space and if you're playing that role, you need to look at three or four different things.
Lisha Rakesh
One is what is your ambition?
Lisha Rakesh
Where do you want to go?
Lisha Rakesh
What are your objectives?
Lisha Rakesh
So, have that value statement very very clearly defined, what you want to achieve.
Lisha Rakesh
And there is again u you know years before the whole idea of automation was based around the ROI.
Lisha Rakesh
What is the ROI and we would be asked uh right uh that what is the ROI that you will bring in if you are uh implementing automation for this project.
Lisha Rakesh
Now this has changed to and this ROI conversation is very difficult because you do the ROI calculation based on certain assumptions.
Lisha Rakesh
Those assumptions never get you know met.
Lisha Rakesh
So when you are giving the ROI uh calculation and you have said that the break even will be after two years two years you are nowhere you know near to the break even.
Lisha Rakesh
Because there is this whole maintenance cost which has skyrocketed you've had multiple different issues and the effort was much higher.
Lisha Rakesh
Because there were certain things that came into picture so it becomes very difficult right so that's the reason why uh the focus has.
Lisha Rakesh
Now shifted from ROI to the business value that is being delivered by automation.
Lisha Rakesh
What is automation trying to achieve?
Lisha Rakesh
What are you trying to achieve with the test automation that you are doing in your project?
Lisha Rakesh
Are you achieving cost reduction?
Lisha Rakesh
Do you want to focus on quality?
Lisha Rakesh
Do you want to you know get the delivery done faster?
Lisha Rakesh
What is the ambition and what is your objective as a leader to implement automation testing?
Lisha Rakesh
The second thing is all around the buyin and the sponsorship.
Lisha Rakesh
You need to ensure that your ambition is clearly called out to the leadership team so that they also understand what are you looking at otherwise you will again get bogged down with these conversations.
Lisha Rakesh
Of we've invested so much where is the ROI when am I going to see uh the benefits coming out of it I'm still seeing the same kind of u issues in production and all.
Lisha Rakesh
Of that so ensure that your leadership understands that there is going to be an upfront investment And there is going to be a certain time before they.
Lisha Rakesh
Actually see things coming out.
Lisha Rakesh
Right? Again, one size doesn't fit all.
Lisha Rakesh
Which means that what works for my organization might not work for you.
Lisha Rakesh
So also ensure that you are being able to understand what is your leadership looking at uh in terms of the objective and whether you are.
Lisha Rakesh
Also aligned with that.
Lisha Rakesh
Then the next one is around the whole automation ecosystem which means that not just looking at the automation platform or not just looking at the automation framework.
Lisha Rakesh
You need to look at everything else.
Lisha Rakesh
You need to ensure that how you you know clearly you are getting your uh requirements.
Lisha Rakesh
Uh what who are the people?
Lisha Rakesh
Do you have the right you might have the state-of-the-art technology in place.
Lisha Rakesh
You might have very very good tools in place but do you have the right people and the right skill set to deliver what you are trying to bring in.
Lisha Rakesh
Most of the times you would see that we talk about very very good um tools very sophisticated very mature tools but you don't find people who can run with those technologies and.
Lisha Rakesh
Then you end up struggling to find the right people.
Lisha Rakesh
Uh you would upskill, cross skill, whatever is needed but it's not always very conducive.
Lisha Rakesh
So look at uh it from that as aspect.
Lisha Rakesh
Look at it from the aspect of environments.
Lisha Rakesh
Look at it from the service virtualization concept whether you because again there is always u this thing that people have especially when you do instrument automation people would expect you to be ready with.
Lisha Rakesh
The automation scripts before even the development code is ready.
Lisha Rakesh
Okay. So look at those kind of concepts.
Lisha Rakesh
Look at uh how well uh you are managing this whole test management part of it.
Lisha Rakesh
Right? Uh how is your uh reporting happening?
Lisha Rakesh
So look at the complete ecosystem and don't just look at it mostly we only look at the tools and the technology but that's not all of it.
Lisha Rakesh
So look at the whole thing.
Lisha Rakesh
Uh dream big start small is something that I always use u uh because what happens is that you would want to maybe create a lot of things and you want to do a lot.
Lisha Rakesh
Of things but uh you know it's always prudent to start with a small P or start small try it out might not work that's.
Lisha Rakesh
Okay you know go back to the drawing board get your team along with you look at what changes you need to bring in if there is a probability to be a little bit of.
Lisha Rakesh
More flex ible in and open to different ideas right make changes try it out again doesn't work go back again sometime it is going to work.
Lisha Rakesh
Right and then you can once you have the right idea implemented it would become very easy for you to take it to the.
Lisha Rakesh
Right kind of the objective that you're looking at right so that that's my uh you know key uh input to you as a leader if you're a tester uh if you're somebody uh who's.
Lisha Rakesh
Working day in day out creating code uh or uh creating the automation scripts, executing them, maintaining our frameworks, working on different platforms etc.
Lisha Rakesh
You need to be aware of the industry changes and trends.
Lisha Rakesh
There's a lot of work that is happening in the automation space and every single day you have with with the whole the this generative AI concept coming to picture there's always a new thing.
Lisha Rakesh
That is coming in right wherein uh people are talking uh so one of our customers came back and said do you still see testing being relevant in the industry.
Lisha Rakesh
Okay so many people have different kinds of ideas so it's very important that you read um what is happening you attend attend uh the industry seminars, webinars etc.
Lisha Rakesh
Talk to people. Uh be uh quite regular and be uh there on the you know professional platforms, exchange ideas.
Lisha Rakesh
You do that you will see that you would be able to understand how the trends are moving.
Lisha Rakesh
It's like stock market right?
Lisha Rakesh
You start working on it, you start observing things, you would be able to make certain predictions.
Lisha Rakesh
It's the same in any of the industry.
Lisha Rakesh
The second uh thing that I want you to know is uh that you need to focus on learning.
Lisha Rakesh
Continuous learning is the way to go.
Lisha Rakesh
Industry is changing. What was there you know when we started was C, C++, uh VB scripting.
Lisha Rakesh
These were hot uh skills.
Lisha Rakesh
Now it's no longer that.
Lisha Rakesh
So we have got uh a lot of different kinds of programming languages that are there.
Lisha Rakesh
So you need to continuously keep on upskilling, cross-sklling yourself.
Lisha Rakesh
The next thing is that mindset of quality over quantity.
Lisha Rakesh
So again like we said 100% automation coverage that's no longer good.
Lisha Rakesh
You need to ensure that you are being able to have the right conversation.
Lisha Rakesh
You bring in that business value concept in picture when you are talking to your stakeholders and you can tell them that you know don't.
Lisha Rakesh
Look at 100% automation.
Lisha Rakesh
We will focus on 20% automation but let's look at these areas which you want to automate.
Lisha Rakesh
We should not really focus on that and that can only come in when you are having the right knowledge of the domain and the industry that you're working in.
Lisha Rakesh
Because you understand the customer you understand the behavior and then the last one is very important we call it the interpersonal skills.
Lisha Rakesh
Because the the expectation of you as a as an automation tester has changed people don't want automation testers they don't want automation engineers they want death they want full stack testers, which means that.
Lisha Rakesh
You need to ensure that you are having the right kind of behavioral skills also, which includes negotiation, which includes being able to interpret things, being creative, problem solving, asking the.
Lisha Rakesh
Right questions, uh being able to articulate things very well.
Lisha Rakesh
So all of those become extremely important.
Lisha Rakesh
Um and and that's my key message to you that you should ensure that you are being able to and you know not just focus on technical skills uh focus on uh you know skills.
Lisha Rakesh
Which are related to your domain.
Lisha Rakesh
Uh focus on the softer aspects of your skill also because when you have an amalgamation of all these different kinds of skills uh only.
Lisha Rakesh
Then you're going to be uh somebody who grows uh in this industry.
Lisha Rakesh
Uh with that uh I think I'm at the end and it's it's just about time.
Lisha Rakesh
Uh so Jeroline over to you um for the Q&A.
Lisha Rakesh
I'm not sure if there are questions.
Lisha Rakesh
So if you can help me.
Jeroline
Sure. Sure. Sure. Lisha and thank you for such a wonderful and presentation.
Jeroline
So like you mentioned it's Q&A time now.
Jeroline
Um so Lisha will answer some questions that were sent to us during the presentation.
Jeroline
And meanwhile audience if you have any more questions please drop them on the uh Q&A chat option that you find at the bottom of your screen.
Jeroline
So here's a question for you Lisha.
Jeroline
So how can we smartly run only limited test cases not full suite of thousand plus test cases?
Jeroline
Is there any um suggestions that you have for this?
Lisha Rakesh
So one of the key ways to do that is do an analysis of your thousand test cases.
Lisha Rakesh
Okay, there's this 80/20 principle that we have which means that most of your problems will be concentrated on some areas.
Lisha Rakesh
If you can identify that and you should be able to because if you're working like I was saying technology is one part but you.
Lisha Rakesh
Also need to understand the functionality.
Lisha Rakesh
So if you can zero it down or nail it down to those 20% areas which are leading to those 80% effects, I think that's a very good way to do that.
Lisha Rakesh
There's also this concept of requirement traceability which means that you are uh ensuring that every requirement has the right test cases mapped to it execution defects etc.
Lisha Rakesh
So that you are being able to understand that if a certain requirement is being changed, what are the test cases that will get changed.
Lisha Rakesh
Right and then what you can do is you can optimize your testing effort because I can completely relate to you we testers are being asked to run everything all the time.
Lisha Rakesh
So these are two things.
Lisha Rakesh
The other thing if you really want to take it to the next level uh so one is identification of those problematic areas and that you can do either you sit with your VAS or.
Lisha Rakesh
You sit with your business you give them your views because you have been running those test cases you understand where the maximum challenges come or you can.
Lisha Rakesh
Also do this production data analysis the production incident analysis or production user behavior analysis to understand what you should focus on more that is one second is understanding what are the key um changes.
Lisha Rakesh
That have come into uh picture for this particular release and then being able to focus on uh those requirements.
Lisha Rakesh
So that you can get a better testing or more effective testing done by running some test cases.
Lisha Rakesh
Got it. I I hope that answered the question.
Jeroline
Right. So there's a follow-up question to that.
Jeroline
So um is there any automated way of doing it or should it be only done manually?
Lisha Rakesh
No, no, there are automated ways of doing it, right?
Lisha Rakesh
Like I was saying that uh we have got this uh this whole AI based uh solution, right?
Lisha Rakesh
Wherein and you can get that implemented as well wherein the production defect or production incident analysis happens uh okay the the data is collected the analysis happens and.
Lisha Rakesh
Then the information gets passed on to us directly.
Lisha Rakesh
So we get to know which are the areas wherein the maximum challenge or maximum incidents were concentrated.
Lisha Rakesh
Now you look at that pattern going forward.
Lisha Rakesh
Of course you might want to do some manual analysis like uh on that uh uh data that you're getting.
Lisha Rakesh
But the data consolidation and analysis is completely automated for us and then you can focus on the manual analysis.
Lisha Rakesh
You can look at it and then you can go back uh and suggest uh to your team members which are the.
Lisha Rakesh
Right uh areas that you should be focusing on.
Lisha Rakesh
Okay. Is there any specific tool that you suggest?
Lisha Rakesh
There are tools um I can come back to you uh I can send this information separately uh on some of the tools.
Lisha Rakesh
Okay. Okay. Fine. Okay.
Lisha Rakesh
Uh so are there any tools or utilities that can be used to implement AI or ML technology in test data management?
Lisha Rakesh
See generally what you would do is you would have um use cases right again there are numerous tools you would find u you know many tools wherein uh the AI uh the use cases.
Lisha Rakesh
The AI use cases are already there these are out of the box and then you can just leverage them and customize but typically what happens is that when you.
Lisha Rakesh
Look at uh because everybody has a different problem right it's it's Not that everybody has the same kind of issues and if your problem statement is different your solution is.
Lisha Rakesh
Also different but there would be uh tools uh for example Tricentus has uh good tools right you can look at that uh that whole platform gives you that uh or enables you to build.
Lisha Rakesh
In that uh a IML capability or artificial uh sorry uh bring in the intelligent automation in terms of the self-healing capabilities or uh you know monitoring understanding what is going wrong.
Lisha Rakesh
Uh so that you can bring in all of this.
Lisha Rakesh
But otherwise what you can also do is you can have something which is your own homegrown uh thing uh wherein you define the problem statement, you define the use case and you would be.
Lisha Rakesh
Able to then take it forward uh by creating the right uh or implementing those use cases uh for your data sets.
Jeroline
Right. Okay. Okay. So here's another question for you Lisha.
Jeroline
So how do you see uh chat GPT and B going to is it going to replace testers or is it going to help testers testers?
Jeroline
What's your thoughts on this?
Lisha Rakesh
See the thing is that uh like I was saying uh this is something that was asked to me by one of my customers.
Lisha Rakesh
Right what what is your point of view on this?
Lisha Rakesh
I think we have to be okay.
Lisha Rakesh
I have personally checked chat GPT.
Lisha Rakesh
I like it very much.
Lisha Rakesh
Okay. Uh because it helps me in a lot of different things uh that uh I want to do.
Lisha Rakesh
Uh the thing is that uh we need to understand again it is in very nent stages I would say.
Lisha Rakesh
Right uh we are still trying to understand how we can leverage that for testing but I don't see that replacing testers.
Lisha Rakesh
Okay, because the intuition that a tester has and like I was saying that interpersonal or behavioral skills, what makes you a tester is.
Lisha Rakesh
Because you have got that intuition, you've got that knack of understanding what goes wrong and that happens because you understand the other users behavior.
Lisha Rakesh
Uh for any generative AI tool uh to come to that level, it's going to take some time.
Lisha Rakesh
It's it's not going to be very very easy uh for them to uh catch up with the intelligence that humans have.
Lisha Rakesh
However, I definitely see them playing a very very important role going forward in terms of different kinds of things.
Lisha Rakesh
So for example, one of the use cases that we are looking at is can I feed a requirement to chart GPT uh or you know of course we are using charge GPD.
Lisha Rakesh
Right now but when we talk in the service organizations uh context we would not typically use an open-source uh uh thing.
Lisha Rakesh
So we would use something which is more secure.
Lisha Rakesh
So if you're using something which is your own homegrown um accelerator or utility uh what we are trying to see is if there is a possibility to utilize this concept to create test cases.
Lisha Rakesh
So if I uh you know ask uh the uh bot by feeding the requirement to them that go to this particular uh uh URL and create a test case for me for reservation that.
Lisha Rakesh
Those things are quite possible.
Lisha Rakesh
Okay. uh however again uh at this point of time we still need people to validate the output that is coming out from them.
Lisha Rakesh
Because these are very uh so you know cutting it short if it is a very complicated kind of a thing I don't think it's the.
Lisha Rakesh
Right solution but if it's a simple kind of a thing or simple applications or simple testing that you have to perform there could be areas wherein you could use uh charg for example we.
Lisha Rakesh
We were trying to do uh a conversion Right?
Lisha Rakesh
So if we want to have a code which is written in a certain language and we want to change it to a different programming language, how do we do that?
Lisha Rakesh
And those kind of use cases are what are coming forward.
Lisha Rakesh
But I really don't think that you know testing or automation testing for that matter will uh be obsolete with these generative AIA tools uh and what's coming into picture.
Jeroline
Okay. All right. So I think we passed time for uh today's webinar.
Jeroline
Um so there's still a lot of questions coming in.
Jeroline
Um but uh we'll get your questions answered through Lisha and share it with you via email.
Jeroline
Uh yeah, since we're running out of time and thank you so much Lisha for answering the answering those questions and for a wonderful presentation.
Jeroline
It was it was a great pleasure to have you with us today.
Jeroline
And uh thank you audience for being very interactive and uh I really hope that you found this webinar valuable and informative and remember keep breaking free and pushing the boundaries of what's possible in.
Jeroline
The QA industry. See you at the next break free session.
Jeroline
Have a great day.
Jeroline
Bye. Thank you everyone.
Jeroline
Thank you for your time.
Jeroline
Appreciate it.
