Shueb Ahmed
All right very good afternoon everyone thank you so much for joining in in today's session which is a part of uh break free series
Shueb Ahmed
So this is a series of talks and we've been asked with the change in leaders in qa series so thank you so much again
Shueb Ahmed
So my name is shaway bahamad and i'm a technical lead pre-sales consultant at v-cloudy
Shueb Ahmed
So in today's session we will discuss about uh the adoption of the cloud testing in a digital transformation journey
Shueb Ahmed
So a quick uh intro uh so where we talk about the how the testing involves when we break free right so testing in software
Shueb Ahmed
Our development has been through several changes over the decades and seen improvements in its efficiency and effectiveness
Shueb Ahmed
So this was possible because of a few minds choose to break free and solve various testing challenges to make it better so now they
Shueb Ahmed
Are here to lead the way so we have taken the liberty to identify and host the change leaders who are transforming the testing space
Shueb Ahmed
For better so the free series but out of conversations around having these change leaders who are ready to share their knowledge and insights to
Shueb Ahmed
Make an impact in the testing space so with with us we have george who is a seasoned automation expert with over two decades of
Shueb Ahmed
Experience in the test automation space so currently he heads the team at test house limited as the vice president solutions he is also an
Shueb Ahmed
Avid test automation evangelist who shares his insights through his podcast channel on automation hangout he has worked in the organizations like usd labs in
Shueb Ahmed
The past and he is running a podcastautomationhangout.com where he basically interviews leaders on topics of automation devops and rpa as per the listen notes
Shueb Ahmed
This podcast is one of the top 10 most popular shows out of 2 lakh to crow 83 lakh 599 and finance podcast globally
Shueb Ahmed
So with that i'll just quickly like to pass this to charge for this quick introduction from the site and then we'll get started with
George
The session over to your judge yep uh thank you thank you uh uh shohei for the quick introduction uh i think you'll have to
George
Pass on the the rights to me so that i can actually share my camera as well
George
I think it ought to make me the host so thank you thank you for the uh for the quick intro so i'll give a
George
Very short introduction from my side i'm georgiou i've been in the software testing industry for the last 22 years mostly working with retail health
George
Insurance and now the banking domain and test automation is one of my core strengths within the software testing stream and as you know like
George
We have we have seen a lot of changes happening in this industry in the last two to three years i would say which is
George
Mostly around maybe starting from march of 2020 that is when we actually started seeing changes in the way testing is actually performed not only
George
Testing the entire i.t landscape has actually gone through a lot of changes in the last i would say like 24 to 30 months and
George
This is primarily because of the lockdown that happened and what i would say like prior to that i would say there is focus on
George
Digital large enterprises especially fortune 500 or the global 2000 companies were mostly focused around building applications that are actually having a digital channel so
George
Visit if it was a retailer it would have been actually good in order to actually have an e-commerce website at the same time they
George
Also had a store and that that is very up that was that is only possible for large retailers especially that's the reason i was
George
Selling like large retailers or large-scale businesses actually had a digital channel whereas many of these small scale as well as medium scale businesses never
George
Even thought about having a digital uh front-end or a solution to sell either the product or the service but things changed all of a
George
Sudden when the physical ways of selling goods and services or products it was not possible
George
So that at that point of time uh even though a lot of service providers like for example the giants like microsoft google google and
George
Amazon was trying to actually push their cloud solution the resistance a lot of organizations and moving to the cloud because they felt that it
George
Was not secure but all of a sudden that adopters people started adopting cloud as well as digital solutions uh to actually sell their products
George
And services so that is one of the important changes that is actually uh found and the other change was basically everyone was dependent on
George
Custom development in the sense like custom development in the sense most of the apps that were getting built were based on what you call
George
By using a programming language or basically it can be java or any kind of language and it was mostly like coded kind of development
George
That used to happen whereas since there was limited time uh to actually push your applications uh into your customers a lot of people started
George
Learning low code no code platforms and organizations also were opening using these technologies to build apps so two changes one is cloud became more
George
What you call people started adopting the cloud second thing is there was a need to actually go digital because of the market situation and
George
Since the resources were not available because everyone was trying to do go digital their resources were very scared was very rare to get and
George
You also had there was scarcity in in the resource pool and then you also had very limited time to push these applications that actually
George
Cost what you call increase that option of low code no code platforms so that is what happened uh maybe in the last uh maybe
George
Last first couple of months during 2020. so how did it impact testing so the way you test the digital application is very different from
George
How you would actually imagine testing a legacy kind of an app that you might have had so the basic difference is the focus around
George
What you call the digital application is not just the functionality where you basically test the requirements or use the stories that is given to
George
You you also have an equal amount of focus around non-functional testing aspects so that's the primary difference that you might you may actually find
George
So let me give you give you some examples of non-functional testing that you may have to actually look out for when you're actually dealing
George
With a digital app so primarily every application that is actually being released if it's a native app mobile app you have to definitely test
George
It across multiple mobile devices and if it's an m website or a mobile website you definitely have to test it across multiple device types
George
Or browser types it can be a desktop-based browser or a browser running on a tablet android on ios so the amount of effort that
George
Is spent on compatibility testing by looking at it mostly like you basically do uh compatibility testing to make sure that your functionality is actually
George
Working across these browsers and mobile devices and you're also worried about the look and feel as well as the experience so you're basically doing
George
A visual validation as well as a user experience validation across these different channels or different devices that you might be having so that actually
George
Takes a lot of your time and there are a lot of solutions that you're actually going to cover today that likely help you to
George
Do this kind of validation the second important aspect that you may need to look at is basically maybe optimizing your website for search engines
George
So people have to come to your website if you want to actually improve your revenue or if you want to provide more services to
George
Your users so search engine optimization is a very key stream even though it was there now it became even more competitive because a lot
George
Of customers a lot of organizations trying to sell the same kind of service so seo testing also picked up because you definitely have to
George
Keep your website optimized so that it actually turns up in search engines or when pre-processing for information it actually turns up that became a
George
Major stream security there was always a focus on security but the number of incidents around security is also increasing i think in 2021 alone
George
We are close to 15 percent of uh what do you say fifteen percent increase uh in the uh what do you say in the
George
Amount of security incidents that happened so uh organizations started uh looking at ways of actually preventing these kind of security attacks at the same
George
Time there's also the dev cyclops culture was also picking up in many organizations so having an independent team around application security was not a
George
Practical kind of a scenario so the devsecops culture was actually what i call improved or people started looking at adopting the devsecops culture within
George
The development organizations in the last 24 months due to which you actually have practices around code scanning architecture reviews dynamics testing dynamic scans and
George
All of those things embedded into agile as well as simply devops teams and then you also had accessibility as another key area of focus
George
Because if you look at the global population close to 1 billion of the global population is actually having some kind of or differently able
George
And if you're not actually supporting your websites or your digital products if you're not catering them to their needs you're actually losing on a
George
Big market share so there is a there is a focus on improving accessibility of applications students say all these applications are external facing it's
George
Important to actually address the needs of the different label people also you might have also seen uh the the amount of workload that is
George
Actually available on the website that existed started increasing because digital was the only channel that people are relying on so websites that are actually
George
Built for maybe for a million users now had to cater to india 2 million so there was increased focus around performance testing where customers
George
Were actually interested in supporting their websites for increased workload at the same time there was also heavy focus on client-side performance so the server-side
George
Performance as well as client-side performance had what you call as an increasing trend because everyone wanted to actually test for these parameters and especially
George
When you're dealing with a website or an app everyone wants the screen to be loaded in maybe two to three second time frame so
George
That also had a major shift and then you also had new types of apps being started rolling out or there was an increase in
George
The number of apps especially around maybe like things like virtual assistants so you had shortage of physical assistance to help people out so the
George
First way of connecting with your customer was actually using virtual assistants and a lot of digital process also were actually getting automated so you
George
Also had an increasing trend around rpa bot implementation and all of these bot implementations were not successful initially and then customers started realizing the
George
Pain points that users are undergoing because when they're interacting with virtual assistants as well as bots and then there are a focus around testing
George
Those apps as well which came up in the uh what they call later stages maybe towards 2020 and 21. and the final area that
George
I would like to touch upon is basically cloud so as i was mentioning earlier cloud was not something that everyone wanted to implement there
George
Were apprehensions about security as well as the viability but since the infrastructure setup was now what you say not possible in the sense like
George
It becomes difficult for people to actually look at what you say doing on-prem kind of application so there was a there was a move
George
Towards uh mowing for cloud-based applications so there was a lot of lift and shipped up applications that were actually happening in the last two
George
Years and applications some of the applications that were built earlier and did not actually support uh what do you say large workload so the
George
Organization started refactoring or modernizing some of their existing apps whereas any new business that is actually starting maybe if you're a startup everyone actually
George
Started adopting the digital first so initially digital first was not something that was actually considered to be a priority but now every organization is
George
Actually going ahead with a digital first kind of what you call objective so that's what has happened and how testing has actually moved from
George
What you say uh traditional ways of testing to more focus around non-functional due to the influence of digital now cloud uh as i impacted
George
As i mentioned like there's a lot of work that's happening around cloud modernization refactoring as well as lipton ship at the same time there
George
Are multiple use cases that can be actually looked at when it comes to cloud so for example it's not just application developers who can
George
Actually use the cloud even test engineers also have room for using some of these cloud-based capabilities i'll actually cover some of these basic capabilities
George
Before we actually move on to the q a session so one is basically around devices as well as what you call browser-based testing so
George
As i was mentioning earlier the amount of browser-based testing or compatibility testing that you have to do is exactly increased there are a number
George
Of browsers and i would say like at least two thousand to three thousand new mobile devices coming into the market every year so you
George
May you may not be able to actually do use all of these devices or bytes all of these physical devices and then do testing
George
In the first place second place you have multiple versions of browsers you have maybe like five or six popular browsers and then multiple versions
George
Of those browsers you you have to test maybe like at least three versions so like it's a big permutation combinations of things that you
George
Need to test and then setting up a local lab or an infrastructure for testing this is actually very painful once you have to spend
George
Money on building it you have to spend time to actually build a lab and then same time you actually have updates coming every now
George
And then for example if you're managing a browser-based test lab your operating system upgrades coming in so you have to actually update your base
George
Infrastructure your browser versions coming in every day so you'd actually keep your infrastructure up to date and running every now and then so the
George
Amount of time that you spend as well as cost that you spend on setting up a lab for cross browser or cross device testing
George
Is pretty high and that's a perfect use case of cloud so you can either set up a device cloud on-prem or you can actually
George
Go with a cross-browser testing or a cross-device testing platform like Pcloudy where you can actually move all of your testing around your digital
George
Applications around compatibility to the cloud the same applies to performance testing as well so you may so previously if some of you have worked
George
On performance testing you you might have you might be testing on maybe a scaled down environment when it comes to performance testing you may
George
Not have all your required infrastructure or the size or size of the infrastructure be very different if you're dealing with a performance testing assignment
George
But now since the input cycle is available in the cloud and you need to use it only for a selected amount of period selected
George
Period of time your performance testing infrastructure is also equally good as your production infrastructure so performance testing is a good what you call area
George
That you can actually move on to the cloud you can actually move your application infrastructure for performance testing or your entire test environment can
George
Be hosted in the core board some functional as far as non-functional testing and you should be able to use some of these what you
George
Call capabilities of the cloud uh to actually scale up and scale down your what you call hardware requirements to do anything like a performance
George
Set so maybe you can actually spin up ten machines to inject or load up maybe a lack of user or half a million uh
George
Kind of users that is one important thing so basically cross browser testing and cross device testing is a good example performance testing is a
George
Very good use case on where you can actually move your application as well as or remove your load injection or your performance testing workload
George
Generation can happen in the cloud your test environments can definitely move on to the cloud for both functional as well as non-functional testing and
George
Finally you can also think about moving your what you say your testing tools to the cloud so if you look at all the modern
George
Tools that you might have for example a tool like jira or most of these tools actually have cloud-based subscriptions so you don't have to
George
Spend time in installing the tool because everything is browser-based so tomorrow if you want to actually maybe there are 10 more testers getting added
George
To your team you can actually go and subscribe for 10 more additional licenses of your product so previously that was not the case if
George
I have a 50 member test organization i may actually go out and buy maybe 60 licenses or maybe assume that i'm buying 50 licenses
George
Maybe only 40 of that is used the remaining licenses are unused because my team doesn't have enough work but now that's not that's very
George
Different you're actually dependent on cloud-based tools you have subscription-based models where you can actually go and subscribe to a license for a month when
George
You actually have work you go and buy those licenses and we are done you can actually leave those licenses you don't have to renew
George
Them anymore so cloud-based tools are also picking up so cloud gives you enormous amount of opportunities in terms of testing oneness you can actually
George
Scale your testing you can do as much of road testing that you want you can actually use in a virtual machines or instances in
George
The cloud to actually run tests in parallel mode you can actually look at reducing your length of testing by actually applying maybe virtual machines
George
Or using virtual machines to do your testing you can improve on your coverage by actually using device based labs or cross browser testing labs
George
And then overall cost of testing also will reduce because you're trying to do things parallelly at the same time you're also looking at buying
George
Licenses on a monthly basis you're reducing on your infrastructure cost so these are some of the benefits that i would actually look at from
George
Cloud and we also covered about what is different in digital so now show hip we can go and take questions from the team
Shueb Ahmed
Thank you so much judge so let me just pick up few questions that are coming in there are some interesting questions that are coming
Shueb Ahmed
So let's let me just pick up so start with uh the first query so the question is what are some of the use cases
Shueb Ahmed
Of a i and ml i should explore in case of application testing
George
You're on mute charge so maybe i'll cover the basic ones so there are at least five to ten use cases that i can think
George
About maybe let's start off with something that is very basic or which is more aligned to digital apps so the first one is basically
George
Around uh maybe keeping knowing your customers so you have a lot of apps that are getting released into your app store or your androids
George
Or your ios store and very important to understand what your customers are talking about your apps so that is a good use case so
George
Getting customer feedback and analyzing the sentiments of the user is a good use case that very can apply because it's not just testing an
George
App you'd also look at the feedback that is coming from your end customers so you can actually have custom applications that can actually look
George
At the feedback or the comments that your users have posted about your product in an app store and then use that information in order
George
To actually improve your testing that's a very simple use case you can also use the data that is actually coming in from your ticketing
George
Tool so your customers would have added data your customers would have actually logged tickets around your product uh in in the cloud in in
George
Your uh production support system your customer support engineers will be actually logging those tickets or in a dpx so you can definitely look at
George
Those tickets and see like what kind of issues are actually getting reported so there might be multiple issues so it can be an issue
George
With respect to the speed of your application it can be an issue that related to your functionality or it can be even a security
George
Related issue or kind of usability issue so if you want to sit and manually do these kind of classification it takes time so we
George
Can again use an ai ml kind of an algorithm in order to classify these defects into various buckets uh basically and using an ai
George
Algorithm you can actually get a lot of inferences the other use case is basically around automation where you can actually use algorithms around self-healing
George
In order to improve uh the way in which you actually maintain your automation script so there are multiple reasons due to which an automation
George
Script would actually fail uh your automation script would basically fail if you're actually dealing with a change in your ui object or it can
George
Be related to a product defect or it can be a workflow change all together let's take an example of a control change that's happening
George
In a what you call a mobile application screen so when you're trying to run these script that is actually using this type of control
George
It actually fails because the object properties have changed so you can actually have self-feeling capabilities embedded within your automation framework or within your automation
George
Tool that can actually fix some of these issues automatically so you don't actually have to go and fix them manually because typically like average
George
Five to ten percent of scripts would actually fail you do these kind of issues and you can eliminate that entire effort by actually depending
George
On self-failing algorithms but you have advanced use cases also like autonomous bots you can actually find an instance of that within the peak cloudy
George
Platform where if you have a mobile app you can actually do at least a good amount of smoke and sanity testing around the app
George
Using autonomous bots so autonomous bots are what you call ai based engines that that has the capability to actually log into your system provided
George
It gives them the credentials it actually goes to every screen do some basic testing and it also has the ability to find out what
George
The workflow is and it actually gives you a report telling like i've run the script on 10 different devices this is how each screen
George
Looked like it gives you screen captures it gives you failure notifications it also tells you like when i was doing testing on these 10
George
Apps this is how my battery and memory performance was so you actually have a lot of options in today's world around automation itself and
George
Finally you also have that it's not just limited to software testing that test engineers are focused on our quality engineering teams are focused on
George
It also has what they call use cases around developer oriented testing so there are tools that actually allows you to do uh test generation
George
Of unit test cases so developers do not have enough time to run uh create unit test cases there are engines or tools that actually
George
Helps you to improve your unit test coverage by helping you write unit tests so there are multiple use cases and but one thing that
George
You need to keep in mind is all of these use cases or products that are available with ai k pool the objective is not
George
To replace humans but they're going to complement your test engineers so that's something that we need to keep in mind so all of these
George
Two tools are basically enables or enablers that likely help a quality engineer to perform his or her tasks in a better way so that
George
Is my response we can go to the next question
Shueb Ahmed
Thank you so much so let me take the next question so the next question is around um which one is better whether it's automation
George
Or many lab testing okay so i would say like it should be a combinational approach so there's nothing like automation alone and that would
George
Work if someone has actually made a comment that 100 automation is possible it may be possible but do we really need 100 automation so
George
You're definitely it's not about writing automation script so i think in the last couple of years one of the trends that i've seen one
George
Of the negative trends that i've seen is like there are engineers who knows how to write good selenium code or java code uh but
George
They don't really know how to analyze a user requirements and then derive test scenarios or test cases from it so that that's a very
George
Dangerous trend so the objective is to actually improve the quality of your uh product whether you're directly doing manual or automated testing and automation
George
Testing if you're actually trying to automate a test case that's written by someone who's actually doing functional testing it'll actually find the effect that
George
Only the manual test engineer is actually or the functional test engineers is actually written so it's not actually going to do something extraordinary to
George
Find defects so the quality of your testing depends on the scenarios that you're trying to automate so automation is not what you call recommended
George
In every possible case it is definitely useful when it comes to testing that is done more than once for example you have to execute
George
The same test case maybe more than two times or three times and use case it can be very different so basically like it may
George
Be what you say testing that you need to do across multiple devices and browsers so i can actually write a single test automation script
George
And i can actually run it on maybe five different devices or a hundred different devices so there's a dedicated use case because you don't
George
Have to repeat the work again and again one test case that is written or one user story that has been written can be actually
George
Automated and then you can run it on multiple devices as well as browsers or it can be a test key that you need to
George
Repeat for every year what you call cycle so a good use case would be like you may not actually do for example your product
George
Has actually gone live you're not making much of changes to the application you may have you may not be actually you should not be
George
Actually spending time on field level validations of every screen that you're having on your application rather i would actually focus on or we should
George
Actually focus on writing tests that actually test a workflow or an end-to-end scenario which can be actually repeated in every build and these are
George
Very good candidates for your system integration test or for your regression test so these are the tests that should be automated so that that
George
Again repeated there are multiple times that you may actually use them or the other interesting area very which does not very much explode because
George
Everyone would like to go on a fancy mode of automation which is ui based api testing is something that you can do maybe 100
George
So you should be actually looking at automating your api testing there are plenty of tools in the market that actually allows you to do
George
That so you should definitely look at automating your api test performance is a very good candidate performance and security you may not be able
George
To do any of these things manually so those are 100 what you call automation kind of services or tests that you can look at
George
But from a functional testing standpoint someone there is some element of manual testing that's required and everything cannot be automated in the sense like
George
It may not be feasible in terms of cost in terms of time that is available so we should have a mix of manual plus
George
Automated testing and the focus will be on product quality whatever mode mode you are using
Shueb Ahmed
Yeah then there's a follow-up question is there a checklist that you recommend to test these apps thoroughly what should be in that checklist okay
George
So from a functional standpoint uh you may not be able to go with the checklist-based kind of approach because the functionality of every app
George
Is going to be very different uh when you move from maybe i'm actually creating an app for maybe paying my credit card bills your
George
Functionality will be very different from what someone else would actually develop but from a non-functional testing stamp and especially when it comes to a
George
Native app there are certain types of testing that is that should be repeated every now and then so that means like it's a standard
George
Set of test cases that you can execute across multiple mobile apps that your organization is building or if you're working for a service provider
George
You can actually use it across customers there are plenty of scenarios like if you actually sit and think through it you may actually get
George
Me like 250 to 300 scenarios that you need to execute on every app so these may be related to simple things like interrupts i'm
George
Using my mobile app and i'm actually getting a video call or an audio call or a text message is it actually causing some kind
George
Of problems for my app or am i getting notifications for my app there might be different types of notifications and settings that are actually
George
Related to my app so these are good two examples and then maybe you can also look at network related aspects so i'm using my
George
App uh can i use my app with both wi-fi and maybe like a data connection is it performing the same way now i'm actually
George
Switching over i'm using my app and i'm switching over from a 4g connection to a lower connection maybe like 3g or i'm actually moving
George
From 4g to maybe wi-fi what happened is my data getting lost or i'm actually in an area where there's not enough reception the network
George
Uh what do you call bandwidth is low the network connectivity is low or i'm in an elevator for a few seconds is my app
George
Crashing during that time so these are things and then they also have device related stuff that you may need to test with for example
George
It can be locking of screen your low battery mode your application what you call your is my application logging out after a few seconds
George
Things like that so i would say like there are a lot of non-functional scenarios uh which can be in the form of a checklist or
George
A set of test cases that you can actually use and this needs to be actually created at your organization so that you can you
George
Actually reduce the time that you take in order to reduce or the time that you take to create these test cases every now and
George
Then one is basically you forget some of these scenarios if you don't have a standard way of testing and then it should not be
George
Like every project i actually spend time in deriving this scenario so these are like reusable test cases that one should be focusing on
Shueb Ahmed
Thank you george let me take the next question so some test automation projects fail some succeed right so what are the things to look
George
For to create a successful test automation project i would say like majority of them fails i would say like 60 to 70 percent of
George
Them would fail and these are due to primary uh very different reasons so everyone needs to keep one thing in mind things that works
George
For me for an amazon or maybe a an organization like maybe a walmart may not work for everyone so the organization culture is different
George
The skill set availability is different the technology might be different so what works for company x may not work for company-wide that's something that
George
Even senior executives or leaders should actually keep in mind the second thing is look at what all applications are there in your landscape and
George
Look at what really needs automation for example if i'm actually going at and modernizing my application next year there's no point in actually doing
George
Automation now you can actually do it after your application is modernized or when you're doing the sport next issue that's very important and then
George
Once you actually pick your up so there should be a framework using which you actually prioritize what needs to be automated from an application
George
Standpoint and once you actually have that in place you should know what areas of that application should be automated so that entire planning exercise
George
Is not done in many organizations so you don't know what you're eclipting the wrong set of applications as well as the wrong set of
George
Functionalities for automation or you might pick the wrong set of test cases so that definitely needs a focus so you definitely need to understand
George
The focus areas of your automation in terms of functionalities as well as applications and then you should also see how much of returns am
George
I going to get and returns always need not be in terms of money savings it can be in multiple ways will i be able
George
To reduce the amount of effort that i'm actually spending on my testing can i release my applications faster can i repurpose some of my
George
Engineers for maybe testing new enhancements so no trends can come in many ways but you definitely need to know what is it i'm going
George
To get when i complete my automation that's very important point number three point number four is around selecting the right set of tools and
George
Framework for automation so that actually is a very important aspect so you definitely need to look at what kind of application do i have
George
So there's no point in actually purchasing a tool that only allows me to do mobile automation if i'm having package applications and custom applications
George
That like websites available in my organization so look at maybe having a very enterprise level uh decision or a set of tools that you
George
Actually have to buy or develop in order to carry out automation so picking the right tool depending on your technology as well as your
George
Future roadmap is very important for you to succeed many organizations i've seen like they invest on a tool because someone thought that the two
George
Would actually do wonders for them and or they'll actually have multiple tools and frameworks being used across the organization which actually causes problems when
George
People are trying to move people from one group to the other because they might not be familiar with the two that is one and
George
The other important aspect where people again go wrong is the approach that you actually select for automation in the sense like there are multiple
George
Ways of doing uh automation you can actually write selenium use maybe an open source technology like selenium write java c sharp or python code
George
In order to automate your test for that you need to have the kind of people in your organization maybe company exactly has that kind
George
Of skill set it may work for them but in my company it may not work because i may not have the right skill set
George
Available so you have to actually pick a call around what kind of automation is required in your organization primarily by looking at what is
George
Available in your organization or what are the type of skills that are available within my organization am i having engineers who are programming savvy
George
Or do i actually have engineers who actually have exposure to maybe a scriptless kind of an automation tool so you need to actually keep
George
That in mind and also there's also a dependency on the type of application that you have so all of these things are very important
George
In a picker tool look at the technologies look at the skills that you have and then decide upon the tool again don't go with
George
What the sales person is telling you or what company x or y has actually done that's very important and then finally when you're running
George
When you're executing an automation project you also have to keep a close watch on what is happening the quality of scripts is also very
George
Important so many times a team is actually formed and the team basically develops a bunch of scripts they they tell you that it works
George
But sometimes when you actually try to run it for a formal regression directly that's where majority of the failure happens in automation because the
George
Amount of money that you spend on automation it never works the assets are not actually tested thoroughly the quality of assets are not good
George
And then finally likely breaks when you try to reuse it so it's very important to keep a track of water that you're developing when
George
The project is on and you need to keep yours you need to actually keep on running your automation skills you can't actually develop thousand
George
Scripts and then keep it in a shelf and then tell that i've automated my application no you should be actually running it frequently and
George
You need to be making sure that exactly in a usable condition the more you run the more it stabilizes so that's also important so
George
If you follow some of these basic things around tools prioritization of applications running bringing in more discipline when you're doing automation development i'm sure
George
That you'll definitely succeed in automation then finally one point that i missed out is ui is not the only thing that you can automate
George
There are plenty of other tasks that you can automate api testing is a good example you should like it may not look fancy because
George
You may not be able to see something happening on your screen that's fine but look at automating different tasks it can actually help you
Shueb Ahmed
To reduce your time or improve coverage that's all have for him all right so the next question is around what are your thoughts about
George
The scriptures automation should companies adopt it and how full proof it is okay there are different thoughts so there are maybe like three different
George
Approaches that you can take as i was mentioning earlier so if you actually have engineers who are good at programming who are technology savvy
George
And if you're actually mostly doing custom application development using java or maybe you don't have to go with the scriptures kind of an approach
George
You can actually go with the scripted kind of approach where you can use a tool a commercial tool that actually supports programming in vbscript
George
Java or c-sharp or python and you can also create a framework around the tool and then start using it and you but for the
George
Prerequisites basically you need to actually have skilled people in your team or you can actually go with a modernized approach uh if you have
George
An rpa tool within your landscape you can very well go and use that rpa tool uh to actually carry out some of your automation
George
Tasks third approach is basically uh completely a pure play what you call scriptless test automation where you can actually go and pick a tool
George
Uh maybe it kavinovsky or it can be a different tool which actually allows you to do scripting or automation without doing a programming language
George
So where i've seen that working i would say like maybe there are three or four things that i've seen working uh in the past
George
One is if you're actually dealing with package applications in your organization most of the testing of package application whether it is sap or whether
George
It is basically a tool like microsoft crm or salesforce or maybe your likely business suit yet smes are actually doing testing and they may
George
Be very good on the domain as well as on the on the functionalities and the capabilities of your package application there may not be
George
People with a lot of programming expertise or coding expertise but they're likely people who knows how to test a workflow or a transaction within
George
The package application so i've seen scriptures adoption or scriptures automation tools working very well with these kind of package applications that's one second area
George
Where i've seen this working is around low code no code development where again
George
You actually get very limited time to develop your scripts because you know code application development or no code application development is almost three or
George
Four times faster than traditional development so the applications are actually turned in very fast you can actually use automation or scriptures to actually create
George
These scripts so scriptless testing is suited for package application as well as the low code kind of scenarios if you do not have the
George
Right skills in your organization if you are if you have functional experts and lists of technology experts it might actually help and if the
George
Speed of development so traditional way of automation using scripts takes time because you have to write code and it also takes time to maintain
George
Them in the sense like it may take time for you to actually update your script so if you're actually looking at accelerating your test
George
Design or your automation design and maybe maintenance time maybe scriptures is actually a good option and many of these scriptless tools that are in
George
The market also has come up with self-healing kind of capabilities so you can actually reduce on very very much on your uh what you
George
Call uh skip maintenance time also but so there are some what you call uh challenges that you may run across when it comes to
George
Scriptures maybe for example you want to build something very custom the controls are not supported or there are not methods you may have to
George
Work with the vendor to actually begin those capabilities and there are i've also seen scriptures tools where you actually have extension kits extension kits
George
In the sense like you can actually write maybe a dot-net code or a java code in order to build in additional capabilities and methods
George
So that also gives you flexibility so flexibility is one aspect that some of the scriptless automation two do not provide but nowadays they have
George
Also considered that as an option so you also have an option to extend your tools capabilities by defining your own custom functions as well
George
As what you call method so that also actually is an added feature so there's no harm in using scriptures it again depends on what
George
Work maybe what works for me may not work for you so you have to actually see what is best but there's definitely an option
George
Also actually having a very good ad option in the market mostly around local no code as well as around package applications that is my
Shueb Ahmed
Take on it thanks george and there are questions around dockers here so let me just read out so what are the benefits of using
Shueb Ahmed
Dockers for test automation or in case of performance testing
George
Okay there are multiple use cases again so docker is not a new thing it has been there for quite some time and maybe the
George
Important use case that i would actually say is like assume that you don't want to depend on a cross browser testing solution maybe like
George
A Pcloudy and you want to actually try run your what you call test automation scripts across multiple browsers so you can actually set
George
Up an input structure maybe in the cloud or on your on-prem where you can actually use dockerized versions of browsers that can actually allow
George
You to run tests in parallel so i can maybe run maybe like i can actually spin up maybe like 50 different docker instances and
George
Then run the test across those 15 senses so there are frameworks or tools like solenoid is a good example there are basically docker images
George
That comes in with browsers where i can actually go and run my test on those docker instances so i don't have to install browsers
George
On my physical machine i can run my test on these what i call browsers which are actually running inside a docker instance or a
George
Container and i can do my test in a very efficient way so maybe like if you want to do parallel testing for cross browser
George
Testing or if you want to run the same test in chrome or maybe like i have i have maybe 5000 tests that i have
George
And i want to run maybe 50 tests in parallel yes i can actually have 58 different docker instances on which i have chrome browser
George
I can run it parallely or the same test i want to run maybe across 10 different browsers so i likely have 10 different very
George
Called docker instances containing different versions of browsers and i can do the execution of these 10 tests in a parallel mode so that's definitely
George
Possible using dockers or the other challenge that i've seen is for example i want to work on a tool and the tool uh for
George
Example i want to do load generation with a tool or a set of tools so i have to install the tool i have to
George
Install the prerequisites so there are a lot of tools that are available now especially around performance and security you can actually get dockerized version
George
So we can actually get maybe a dockerized version of jmeter and then start using it or i can actually install maybe a devops tool
George
Like jenkins within a docker instance and then i do nothing i don't have to install anything i have to just download the docker image
George
And then spin an instance using a command and then start using my jenkins instance so the installation speed the pain that you have to
George
Undergo installing apps all of those things can be actually solved so cross browser testing or maybe compatibility testing or parallel testing is one you
George
Second use case compatibility can be the first use case having dockerized versions of tools is maybe a third use case that i can actually
George
Look at and then maybe you can also have your application if your application supports containerization you don't have to worry too much about installing
George
Your application your involvement so what all copies you have created in your test environment the same docker image can be actually used to create
George
A pre-production environment so application deployments also get faster and it's also easy to integrate all of these things into a devops pipeline for example
George
I want to create a pipeline for running my security test i can actually have all my security tools in dockers and i can actually
George
Spin them up and then what you can call them inside my ml script or my ci cd script and then use them for my
George
Uh what you call devops so there are multiple use cases exactly something that you definitely need to explore it saves a lot of time
George
And pains associated with the installation of application is relatively faster and also allows you to scale your testing to any extent possible you can
George
Actually run parallel tests and then complete your test cycles at a very short duration just some of the simple use cases that i can
Shueb Ahmed
Think about thanks judge and a few more queries coming in so how does cloud testing help organization cut down their capital investment
George
Okay so i touched upon it at a very high level so maybe the simple example is if i want to do maybe cross-device testing
George
Or if i want to actually set up do a testing of a mobile app let's take an example of that so i may actually
George
Have to maybe buy mobile devices and i was selling like a 2000 3000 devices coming into the market and it may be very tough
George
To actually identify like uh but uh you'll have to actually look at the worst case scenario allowed to test with 10 devices so i'd
George
Actually buy 10 devices you will buy 10 devices now and tomorrow maybe after six months you have another five devices coming into the market
George
So i buy those five devices so you have to keep on buying devices to do your testing that is one now the current scenario
George
I don't think any organization so we actually have distributed teams so there might be people working from home there might be people working from
George
Remote locations and if you're actually working in a mobile project i can't actually buy devices for everyone and it's not easy for me to
George
Ship devices from one location to the other because you have very short time duration for your test cycle so it may not be practically
George
Feasible to actually buy devices for everyone so you can definitely reduce on the cost of your devices by actually going ahead with what you
George
Call uh device pla farm in the cloud or in a public cloud or on a private cloud depending on the type of need that
George
You have the same applies to cross browser testing also it's very tough for me to actually buy hardware where i can actually support testing
George
On multiple browsers or even if i want to actually run maybe 50 tests on my infrastructure i need to have that kind of capacity
George
You know if i'm using dockers to run that 50 instances docker instances to do parallel testing or cross browser testing and the other pain
George
Point is i need to have someone dedicated to managing these devices and browsers because these they they come up with updates the vendors would
George
Actually come up with updates every now and then so you have to actually invest investigate enough physique in a person also to manage your
George
Lab exactly adds up to the cost rather it'll actually move my entire testing of browsers as well as devices to the cloud where i
George
Can actually look at what you call reducing my cost of infrastructure as well as the cost of maintenance the same applies to performance testing
George
If i want to run a test with maybe like a million users in my uh on-prem kind of an infrastructure it's actually going to
George
Cost me a lot i would actually buy hardware devices that likely support my maybe even if i'm going with a scaled-down version of my
George
Production environment it's actually going to cost me a lot in terms of infrastructure costs so it actually is actually going to add up to
George
My cost whereas if i'm going ahead with the cloud-based what i call this one i don't have to buy any kind of infrastructure so
George
I'm not investing in anything i might only pay the ongoing expense whenever i'm actually doing performance testing or actually pay for that environment and
George
After my needs i'll actually shut it down and finally and another way of reducing your cost is maybe by looking at reducing your investments
George
Initial investments on tools so if i'm uh if i'm having 20 if i'm having an aid of 15 maybe two licenses maybe a test
George
Management or automation licenses i'll only buy those licenses now tomorrow if i'm actually scaling up my project team i'm adding another five people uh
George
To my project maybe in the month of august i'll buy licenses only for the month of august and september and after that i know
George
That they're actually going to come out of the project i'm only going to invest in rules for only two months again you're actually buying
George
It on a what you call subscription based mode rather than actually buying maybe 15 or 20 licenses up print and keeping it in my
George
Shell so i likely go with that kind of model so all of these things together will actually help you to reduce your initial investment
George
So you can actually go with an on-demand model whenever you have a need you go and invest in both infrastructure as well as two
George
Licenses so infrastructure is mostly around using the cloud whereas two licenses you may actually go with a sas based platform where you can actually
George
Buy licenses on a monthly more in subscription so that is what i would recommend
Shueb Ahmed
All right so let me just take the next question so i'm a functional test engineer and how can i shift my career to the
George
Automation stream okay so so i think there is a general perception that functional testing and test automation are the only streams that you can
George
Actually build a carrier in okay that is in fact wrong or maybe there's another perception that if i am a tesla the only option
George
For me to actually is to get into test management that is actually a false perception so there are a lot of things within testing
George
If you're actually doing functional testing and there are a lot of streams that you can actually get into if you want to actually move
George
Away from a manual mode of testing you can definitely pick maybe automation and you don't have to be programming savvy in order to get
George
Into automation exactly mentioning earlier the scripted kind of automation is just one mode of automation you can actually go ahead and learn a tool
George
Like kopsky or try some dystocia and then become a master in scriptless automation or you can actually go and learn a tool like ui
George
Path where you can actually build expertise in rpa as well as on rpa based testing or using rpa tools for testing so you don't
George
Have to be programming savvy in order to shift your career towards automation but it's good definitely to learn a programming language there's a lot
George
Of opportunities on the open source side as well so maybe the top tools that if you want actually guru god and do scripted kind
George
Of testing you can actually go out and learn tools like selenium with java or c-sharp as the programming language or if you are starting
George
Something now maybe you have to actually look at the modern age tools like playwright for example or a cypress which is mostly focused around
George
Angular based automation that's one stream of tools that you can learn and you can also focus on api based tools maybe to start with
George
You can actually start with maybe something like a karate framework rest tester or postman those kind of tools so definitely there are options they're
George
Going with script lists most of these vendors our tool providers actually have beautiful training academies that you can use the same applies to rp
George
Also that that likely help you to gain expertise around these tools and the integrities around these tools okay and there are a lot of
George
Material available in the public domain youtube is a very good source i've seen three channels that actually allows you to do uh what you
George
Call uh learn test automation other things and then you also have good amount of courses in platforms like linkedin learning as well as go
George
To me so there are plenty of free resources as well as paid resources using which you can actually learn test automation the only thing
George
That you need to have is confidence as well as a passion to learn that's where i i don't see both of these things in
George
Many people so they they feel they don't have the confidence that they can learn a tool nor they are ready to put an effort
George
Nor they have a passion to learn it so then no one can help you you'll always continue to do testing and the man will
George
Want and then if you are not a person who if you feel that you are not the right person for automation you do not
George
Feel bad there are other streams within testing where you can build a carrier i look at maybe data data is a very good field
George
Everyone is struggling to get what you say proper test data for their what you call projects so you can look at learning taking up
George
Test data management as your stream or if you're actually good in writing sql statement they've done data warehousing database testing look at building a
George
Carrier around data testing you can actually be known for as a data testing professional there's a huge demand for good data testers or maybe
George
Performance might be the field for you if you're someone who's actually playing around with uh maybe things like infrastructure you're a fond of uh
George
Performance or maybe like uh the hardware part of it as well as application performance performance engineering might be the right what i call skill
George
For you it is something that you can also master again there are plenty of resources provided by vendors like placement and others where you
George
Can actually learn all of those things cyber security you can actually focus on application security or the entire network security as a stream they'll
George
Have plenty of opportunities around that test architecture is a good sleep you can actually be a test architect if you're a lead we don't
George
Want to get in test management you have to actually look at becoming a test architect or an automation architect or a performance architect a
George
Lot of streams and possibilities but you definitely need to explore automation not this is scene and with an automation learning programming knowledge is not
George
A pre-requisite to become an automation engineer anymore you can either learn on a script plus tool or what you call an rpa tool and
George
Then still do test automation or you can actually focus on api based automation where you don't have to do a lot of programming but
George
If you're someone who's very fond of programming definitely you can go with the open close route learn java or use your java or python
George
Or c-sharp skills to build tests so there are multiple options and there are plenty of resources available in the public domain what do you
George
Mean is there there are plenty of mentors that you can actually go after in using a platform like linkedin will be ready to help
George
You for free so there are options only thing that you need to have is a passion confident that you can learn things and you
George
May have to spend some amount of time to actually learn so dedication dedicated time is also required to learn these or acquire these skills
George
But it's important to stay relevant uh in in this theme because the testing as a stream is actually going more it's more mostly becoming
George
Technology oriented for example you need to learn technology in order to even at least if you want a good example if i'm testing an
George
App in the cloud in the azure environment or if i'm actually testing doing a performance system open app that is actually hosted in cloud
George
I should be knowing the basics of cloud for example if i'm a performance tester i should know the basics of azure so i'm running
George
My performance test in azure i should be knowing how to actually monitor the performance of my azure app using azure monitoring services and things
George
Like that so you shouldn't become more technology focused as well it's not just focusing on testing you should also learn some of these technologies
George
That are actually come into the market that's a that's a long answer
Shueb Ahmed
Thank you so much for patiently answering that query judge i'll just take few more questions we are nearing to the end of the session
Shueb Ahmed
But like just put some more queries here so there's one quick question so how do i select the right set of uh browsers for
George
Testing my website uh very interesting questions as there are multiple browsers and multiple what do you say devices available in the market it's very
George
Tough to take a call but i've seen reports for example i've seen a report from peak cloud itself that actually gives me a fair
George
View on what kind of mobile devices are being used and you can also get similar reports around browsers where i can go and find
George
Out for example if i'm releasing my product or website to the public domain for the first time it will be very easy for me
George
To actually go and look at some of these reports and then decide okay this is this is the type of users for example i'm
George
Dealing with this app and maybe in a country like maybe singapore i should be able to go and find this data around what kind
George
Of mobile devices are being used by people what kind of websites are being used by browsers are being are going to be used by
George
People and then do my testing around those websites or around those browsers and devices because i do not have any data available because your
George
App is new or your website is new so you have to look at what is available in the public domain it can be reports
George
That are published in the public domain or it can be a platform provider specific report like the one that i was talking about cloudy
George
They can actually give you that initial insight now once your application is actually being pushed into production look at the website look at something
George
Like google analytics it gives you a wealth of information around things like uh the web the operating system from where your customers are actually
George
Logging in what pages are being used by your users what kind of resolutions are they using what kind of browser versions they are using
George
All of those things are actually known to you but rarely have seen testers going and exploring these things our objective was like my project
George
Manager told me that i would test in five browsers i'll decide what are those five browsers no if your app is already there in
George
Production look at some of these data that is actually coming in that basically again an area of production feedback where you can actually gather
George
This data from your application logs or maybe google analytics use the data to decide upon what are the five browsers or devices on which
George
I should be doing my testing because you definitely see a trend maybe like 60 or 70 personal your users are actually using either firefox
George
Or chrome and they're only very like two to three percent using it so will i prioritize on it or would i use chrome and
George
Firefox definitely i'll go with chrome and firefox and then maybe do some basic tests on edge to make sure that it actually is working
George
So you have to actually adopt as base testing approach when it comes to cross browser and cross devices because there are plenty of browser
George
Versions browser what you call providers and devices that are available in the market and you cannot do testing on all of those but definitely
George
Go with the risk-based approach where you only look at the important ones rather than actually doing and spending time on what you call testing
George
Everything so that is not the right approach you'll have to definitely adopt this base approach but the risk-based approach should be definitely what you
George
Call backed up by enough uh metrics or data that is available in public domain as well as from your own uh what they call
George
Servers if it is like a website that is currently available
Shueb Ahmed
Thank you and just try to take one more question here so what are some of the metrics that will help us to demonstrate the
George
Value of test automation uh okay so that's a very important thing especially i think this is more applicable to senior leaders in our organization
George
Because it's very important to show that automation is working in your organization so it should not be like they can depend on your goal
George
So the goal of automation is not just to reduce cost sometimes it can be but it's not always so it can be in terms
George
Of uh bringing in more coverage or it can be in improving the throughput of your releases for example i'm doing three releases now i
George
Want to actually what you call make it five releases so how can i do that i can actually reduce the time that i'm taking
George
For my testing by bringing in more automation for example if one test cycle is actually taking 15 days for me can i make it
George
Three or four days so this depends on your objectives and some of the simple metrics that they can actually use which will actually help
George
Both stakeholders from business and i.t siders may be like look at your release frequency how frequently am i releasing now am i actually having
George
Maybe like releases every quarter or am i putting maybe number of releases can be good example uh previously maybe before automation is actually doing
George
Three releases now i'm actually doing maybe six releases because i'm actually cutting down on my test cycle or go one level below that look
George
At the amount of time that is taking for your test cycle i was taking three days for doing my test cycle earlier now i
George
May able to do it in maybe half an hour or one hour so test cycle duration before adapter can be a good matrix amount
George
Of effort or cost that you're spending on test automation before and after or on regression testing is a good example for example i was
George
Actually taking maybe like thousand hours to complete my regression testing earlier now i'm able to actually complete regression testing maybe in 100 hours of
George
Effort so these are some of the simple metrics or it can be in terms of quality your once you started doing automation once you
George
Started running your automation scripts across multiple browsers and devices your quality has improved you want to demonstrate the quality improvement look at how many
George
Defects were actually getting reported earlier because of compatibility issues and now once you implement automation how many of them are actually getting reported now
George
So these are simple metrics but very rarely leaders or managers do these kind of reporting but it's very important to actually demonstrate the value
George
Of automation to stakeholders because they're investing money in automation so it's important to show that it is working in your organization sometimes automation may
George
Not be working for you and it's and you unless you actually have data to prove that it is working or not working it's not
George
Going to help anyone either you have to stop investing money and then focus on the right areas or automation or if it's working you
George
May actually try to do more automation so it's good to take you need to have some kind of metrics getting reported on a weekly
George
Basis or on a monthly basis so take a call and also demonstrate the value it is very important
Shueb Ahmed
Well thank you so much uh judge uh for your sharing your valuable insights with some practical examples and also the real world scenarios i
Shueb Ahmed
Really appreciate your time and uh here certainly your answers will have definitely covered a lot of practical pointers and you know provided our audience
Shueb Ahmed
An insightful experience and also the knowledge right so they which they can use and apply so it's a pretty honor to be hearing from
George
You thank you again thank you so much for your time
George
And we also thank you cloudy team for the opportunity to it was a great set of questions and i look forward to more of
Shueb Ahmed
Such events in the future thanks judge and we also want to thank each one of our participants for joining us at our second webinar
Shueb Ahmed
On the breakthrough series stage uh so we are sure george inside of the on the adoption of cloud solutions and also the automation has
Shueb Ahmed
Parked few ideas that you can take back and implement for yourself so thank you once again for being a wonderful audience uh good day
Shueb Ahmed
Also please feel free to write us at info claudia.com with your questions so we will try to reach you out personally as soon as
Shueb Ahmed
Possible and just a bit of note uh Pcloudy is organizing a hackathon for all the automation users so the automation is planned on
Shueb Ahmed
30th of july you can still register on the platform please log into piclody.com
Shueb Ahmed
And you know just run through the hackathon the plan that is set out so there are a lot of goodies and exciting prices to
Shueb Ahmed
Be won so look forward to see you there as well so thank you again thank you so much for your time and uh thank
Shueb Ahmed
You george thank you thank you team have a great day bye bye have a great day bye
Shueb Ahmed
You
