Jeroline
Hello everyone and welcome to the first episode of the Year 2023 of the change leader series Break Free so are you ready to take your test automation game to the next level because today we'll be looking at the topic three-dimensional test automation strategy my name is Geralyn and I'll be your host for today so before we get started I would like to go over a few housekeeping rules your line is currently muted but don't worry you can submit your questions during the webinar through the share option that you see at the bottom of your screen we'll answer your question as a part of the Q&A section at the end of the webinar
Jeroline
And also please note that this webinar is currently being recorded and we'll send out the link of the recording so that you can have a watch later or share it with your friends affiliates so with that being said let me quickly share a bit about the Break Free Series so Break Free was an idea that births out of the need for change in the testing domain so we've seen the testing industry evolve and grow over the decade as a few leaders took the risk to Break Free from traditional approaches towards testing and come up with new ideas and practices so we believe that this change can be accelerated at a much faster pace so we took the liberty of identifying and inviting a few change leaders to Breaking the mold and embracing change to see the QA industry
Jeroline
Work so that's what Break Free series is all about and we hope that these sessions would spark ideas in each one of you to be the change that's needed in the QA industry so the industry itself grows into greater leaps and bounds so with that let me welcome the change leader for today is a quality engineering lead having 10 plus years of Rich experience in the software testing industry is currently an engineering manager at a growing status Zuppie he hits a team of quality engineers and handles key automation projects He has a passion for quality and is an active contributor in the QA Community you will find him talking about his experiences and sharing best practices in various conferences and other events he also enjoys learning from other experts and community members during the part of
Jeroline
This journey so without any further Ado let me hand it over to the speaker for today sahil Pune
Sahil Puri
So hi everyone a very good evening and welcome to the session first of all I would start with thanking Pcloudy and the team for such a humble introduction and also for this opportunity to share my experiences and learnings with everyone today through this webinar so I think as already introduced and sahil having 10 plus years of experience in software testing industry now what drives me in this role is the love and passion for quality and user experience I'm currently working as an engineering manager for a growing startup company where I'm leading and managing a team of around 30 engineers so that with that being said without further delay let's start with today's session I think we have a good number of audience so let's have a simple look on today's topic just oh yeah so what exactly is a multi-dimensional
Sahil Puri
Structure right so I think all of us would have studied the concept of dimensions in our school or college Years which says that dimensions are a measure of size or distance of an object in one direction now the number of Dimensions which are used to represent a figure makes it an n-dimensional figure for example a dot it neither has a length nor a breadth so it's a zero dimensional figure again a line requires one dimension for its representation so it's a one 1D figure similarly A square has two Dimensions length and breadth represented by x and y axis also a cube has another dimension added in the form of height Z axis which makes it a three dimensional figure now this concept of multiple Dimensions
Sahil Puri
Can be applied to anything and everything having multiple aspects all dimensions now it can be a figure a real structure a business marketing or even a development strategy so on this note a multi-dimensional strategy is one which requires a holistic view of all the aspects in order to create the desired venue now this increased complexity of systems processes consumer behavior digitization in today's world have led to the requirement of a multi-dimensional strategy now based on this definition and what we have discussed till now I want everyone of you to give a thought to my next question
Sahil Puri
It says is test automation strategy multi-dimensional now hold on to that thought we'll come back to this question again very shortly but let's first have a look at some facts which might help you to reach a conclusion so the U.S testing industry is worth close to seven billion dollars by the end of 2022. with global budget reaching 40 to 50 billion dollar Mark and the figure in India itself reaching 8 billion mark globally IT industry spends around 28 of its total budget on testing with a special push on automation
Sahil Puri
Over 60 percent of the teams have achieved more than 50 automation of their current test and related processes and around 62 percent organizations have increased their yearly budget on testing as compared to last year now these are all the good news for testing and automation community but here's the last fact which requires some brainstorming which says that only 14 of product based organizations and nine percent of service-based organizations achieve close to or more than 80 percent of desired ROI that is return on investment on the automation efforts
Sahil Puri
Now there must be some serious Gap that even after so much of budget allocation and so much of prioritization only a specific percentage of teams are still satisfied and a major proportion remains dissatisfied with their automation effort efforts and automation strategies right well let's also look at some of the survey results that I managed to gather for the biggest struggle and pain points in using test automation today now these problems or blockers as you can see on your screen vary from lack of training hiring challenges finding the right tooling difficulty in setting the right process
Sahil Puri
And approach across the organization lack of standing report standard reporting or metrics availability budget allocation problems two requirement changing too often less or no time for test automation sometimes the senior management is not supportive enough of our vision related to automation some technical problems related to test data and environment setup again depending on the level of DevOps in the in your current organization now the lack of skilled resources lack of mobile devices available for Automation and so on now what comes first to your mind after looking at these survey results in the last two slides yes I know what you're thinking I'm also Amazed by such a wide range of challenges faced by teams across the globe while building or starting a test
Sahil Puri
Automation project right so coming back to my original question what do you all think is test automation strategy multi-dimensional yes I think we all can agree especially given by the last two slides where we saw a wide range of challenges that are faced by Engineers Like Us in building the test automation project or test automation system in our organizations and especially seeing the variety of challenges which are observed in each of these steps it surely involves multiple aspects and multiple Dimensions hence it surely comes Under the Umbrella of a multi-dimensional strategy
Sahil Puri
Right now in a very generic sense an efficient or test automation strategy or for that matter any strategy it revolves around the 5w and one Edge that is the why what who when where and how of automation so this approach pretty much solves all the strategy and planning problems in every space so why not automation yes it does but it is a very generic plan and a generic strategy but let's go deeper today and let's view the same problem from a bit different approach and angle and what's that different angle so let's now jump to a three-dimensional test automation
Sahil Puri
Strategy that we are going to discuss today so starting with the First Dimension the foundation now by the laws of construction the foundation is the most important step in designing a hundred story state-of-the-art building in terms of test automation strategy the foundation or the First Dimension refers to clearing the basics of testing and understanding the current state of testing and quality in your team and across the organization so please note we started with understanding the current state of testing and quality in the team right it starts with observation this would mean how quality is perceived by every individual directly or indirectly involved in the SDLC that is the software development life cycle
Sahil Puri
So you need to identify the current test cases test Suites test processes so that you can smoothly integrate your automated scripts and pipelines in the current running processes or at least the Improvement areas and slowly start to tweak the processes accordingly now these points are very much necessary I think there is no doubt they are very important because when you are just starting to build an automation project it is not at all advisable to change all the current processes in a single go because this can draw repulsion and concerns from entire organization and multiple teams so the right approach is to understand the current state identify the good points
Sahil Puri
As well as the possible loopholes of this current state and then build upon this data in your research to propose moderate level changes to bring the entire vision of automation into the whole picture next it also requires you to identify the current levels of testing which are covered across the SDLC and its specific coverage this can be code coverage test coverage or any other coverage metric but this normally includes what kind of test and validations are developers running at par as part of their development process like unit test integration test
Sahil Puri
Then what tests are run by QA or QE team in the testing phase similarly what all tests are part of release Engineering Process or the release engine team which handles the entire release cycle and finally what kind of test Suites are run as part of post-productions and deployment sanity Suites I'll give you an example so if the if your developers or the development team are already developing and running unit test and integration test as part of their development cycle then you can easily collaborate with them and understand the code coverage of their unit test so that you can now focus on more in-depth test cases while testing and automation to avoid the redundancy of the coverage
Sahil Puri
Okay going to the next point which says understand your application which you are about to automate so dig into the product flows based on the user journeys for this you need to sit with your product team business team and maybe data teams in your organization to understand the P0 user journeys or the golden user flows of your application similarly put on your debugging hat and get an understanding on the end-to-end architecture of your application now this includes the internals of the system or what is happening behind the UI say normally I would say there are multiple layers in an application if we see the end-to-end internals of the system starting with the UI the middleware where normally the authentication authorization rate limiting and load balancing happens then
Sahil Puri
Comes the micro service layer or I would say the brain offer application followed by the database layer which is normally the storage layer but there might be a websocket layer for TCP connections a cache implementation to optimize the DB calls and maybe some queues for asynchronous communication now all of this understanding of the system internals would really help you to identify the layers you want to test with different types of automation and what automation coverage is best for which type of scenario and test data let's take an example a typical case of a sign-in flow so suppose you have a set of 15 test data inputs that you want to test and automate for this sign in flow
Sahil Puri
Now generally without going into of the behind the curtains and what's happening in the internals of the system and without knowing the layers of this our application you might end up automating all the 15 test inputs from the UI itself which would take forever to run and maintain however there can be an efficient approach now if the deeper layers of your system and design an automation strategy in accordance with those layers you will just automate maximum two or three test cases from the UI layer and rest all the cases you will automate on the API or much deeper layers going till the database layer thereby increasing the efficiency and maintainability of your automation system
Sahil Puri
Thank you next it is advisable to get deeper into user journeys and usage Analytics yes it is true for automation along with your normal test plans now the more you understand your user the better you will be able to optimize your Automation and overall testing strategies now one example for say in today's digitized world the probable range of devices and operating systems in production used by our user base have increased exponentially like Android 6 to 13 across the range of OEM or manufacturers iOS devices web OS tizen Chrome Etc so understanding the geographical distribution of your user base the device platform and operating system on which your user is actually running your application in production will really
Sahil Puri
Help you in designing a better and efficient automation Charter here is where some of the latest AI powered end-to-end Cloud Solutions like Pcloudy can really help you with these optimizations and ensuring that you are covering the entire Horizon of devices and operating system when running your test and automation Suites right next keep a close eye on production stability and production incidents now for this you might want to meet with your customer support team and get the data on all the support tickets and their frequency on a weekly or a monthly basis
Sahil Puri
Now understanding the production incidence and user experience issues is one of the most crucial metric which will guide you on what kind of scenarios you really want to automate and where to prioritize your efforts on early on in the project because our ultimate goal is to increase the product stability and give our user the best possible experience for example if you have your application hosted on say Google Play or Apple App Store then you can try working with some of the store developers apis and inject the user reviews on runtime to a machine learning based categorization algorithm which will simply categorize all the reviews in multiple buckets and help everyone in the organization including you and product even Tech Team to understand the user's top most pain points for all the one or two low star reviews of course you can filter the stop words
Sahil Puri
To take only the constructive feedback into consideration but again try to stay closer to your user and related production data while deciding what to automate so as they rightly say if you automate a mess you get an automated mess right so the first task is to identify what's messy in the current system without directly jumping to Automation and start to automate everything and anything it is very important that your automation strategy includes this point that is the First Dimension since it will slowly unravel the current state of things for you and your team and basically make you ready for the next coming dimensions
Sahil Puri
So a quick recap in the First Dimension we were laying a foundation stone and a strong base for understanding the application the test cases and the current state of test processes and the current user experience problems that is faced by your users on production which majorly covers the question of what to automate right now jumping to Second Dimension which says automation system and tools okay so let's go deeper into the second dimension
Sahil Puri
Now first and foremost you have to consider your automation project as an end-to-end Software System or a development project then it is advisable to start gaining knowledge about different available tools which will help you to take an informed decision around the selection of these specific tools as per your situation and condition now this knowledge can be developed by planning and executing some POC that is proof of concept followed by a group demo to chalk out the pros and cons of different available tools in the market and plan brainstorming discussions with the entire team to take everyone's past experience
Sahil Puri
Into account but the most important thing is to explore and compare a set of tools which you feel are the right fit for your use case and your situation right then comes the high level design of the entire automation system which basically covers a block level map of the entire system's control and data flow so this Dimension also covers a detailed framework design and the design patterns to be used with the tech stack of the entire automation system now this would normally cover the programming language build tools the test framework like JUnit TestNG just CI/CD tools
Sahil Puri
Reporting dashboards tools for monitoring alerting and some Cloud tools Etc next we come to some of the problems that need to be discussed and solved at this phase in order to increase the maintainability and scalability of our framework later on so the two pillars of this dimension are the maintainability and the scalability of our automation system so you need to strategize and plan out different ways to generate test data and make our test independent of the environment that they are running on now I have seen cases where the automation is tightly coupled with the box or environment it is running on like staging box or a QA environment thereby leading to flakiness when running on other environments and not allowing its Global usage across the teams right so you cannot
Sahil Puri
Keep the automation to yourself or to specific to a box or an environment you have to let it become Global so that it can be used across the teams and for that you need test data Independence then comes the planning around feasibility of parallel test execution to save time and resources now this might sound like a small problem in the beginning but believe me this is one of the toughest nuts or things to crack especially for UI automation now this would require a solution at both your framework and the device lab end so it becomes a very important problem to plan and strategize especially early on in your strategy
Sahil Puri
Right the next is the problem of result reporting but the result reporting structure the format frequency and the channels on which you communicate these reports like Slack email should be logged in this Dimension to form Clarity for the actual development execution and the coming phases of your strategy now without a proper result report your automation truly remains unexplored especially by the management again a very important part of this dimension
Sahil Puri
Is setting up your own test environment or the box which includes your knowledge on code deployment configuration setup database changes thereby requiring a Mastery On containerization Tools like Docker Kubernetes and some of the cloud systems like AWS as your GCP no monitoring these environments boxes and setting up alerts for any anomaly inbox parameters is also a part of this Dimension and your RNR now this normally helps to ensure that automation pipelines are up and running in a healthy State and you are alerted as soon as the Box requires any kind of maintenance now you can try out some of the tools
Sahil Puri
Like Prometheus Sentry etc for this kind of environment Health monitoring and alerting and next coming to one of my favorite points which says automation analytics and designing analytical dashboards now this is one of the most undervalued points in automation strategy which leads to lack of visibility to higher management and affects the data points available to showcase the improvements in your automation system it is always advisable to build analytical dashboards now if you look at the reason is you might want to share monthly or quarterly reports of your Automation and show the Improvement in flakiness or say the to showcase the Improvement or the increase in the test coverage or
Sahil Puri
The code coverage of your automation scripts right and for all these reports you need some analytical or historical data so you have to think through of all these requirements in your initial strategy otherwise it becomes very late on the actual execution phase then comes a very important Point flaky test you have to develop a strategy to identify stabilize and monitor your flaky test cases now the flakiness is that bug which eats the entire credibility of your automation system if left unattended but I don't mean that you have to have any sort of fear from flakiness you have to befriend the flaky test
Sahil Puri
Cases but make sure that it remain it doesn't remain unattended that you are taking some or the other step on your flaky test as in when they occur and don't leave them to rot otherwise it will really eat up the entire credibility of the automation system that you have built with your hard work so this Dimension should cover the steps taken to keep flakiness in check now this can also include Building Systems that auto detect some of the flaky test and do not run them in subsequent builds until an action is taken on these flaky test cases
Sahil Puri
Then coming to logging and debuggability now these are the two most important points or tools required to reduce the time which is spent on analyzing automation failures and providing a possible RCA and subsequently give a green signal to the development team or the release team in case of false negative results right again it amounts to the trust and The credibility of your automation system so you would always want to have the highest possible levels of logging and debug ability so that any kind of false negative does not become a reason of delayed releases and your trust and credibility of the system remains High then we need a strategy to get rid of noisy and redundant test
Sahil Puri
Now what are noisy automated tests now noisy test can be the ones that require a lot of data setup can come under the category of can be automated but should not be automated or simply cause the company some money every time it is triggered on automation so the process and system require regular refactoring of these automated cases this amount to the much required maintenance of the system and automated scripts which need to be efficiently planned in this dimension last but not the least designing of CI/CD pipelines now this is kind of a buzzword and a really important thing to consider while strategizing is strategizing your automation project
Sahil Puri
CI and CD actually provides last-mile connectivity to all the potential automation users by reducing the dependency and solving the problem of when and where to run our automation now CI pipelines are really useful to collaborate with developers and run our tests on every pull request or a code merge thereby alerting developers of any faulty PR as soon as it is merged right so it ensures early feedback as well as ensure that your automation is running and becoming coming to use by the development team as well now this strategy is very important to avoid a situation where automation or a QA team is running the automation
Sahil Puri
Scripts only once in twice or so three weeks at the time of releases in the form of regression test now running the automation only for regression solves very less purpose and it doesn't leave a mark unless used in the right places and across the organization similarly The Continuous delivery and continuous deployment pipelines have to be designed and built to run your automation Suites and ensure a shippable software ready maybe at the end of every hour or even at the end of every day now this is the basic difference between continuous delivery and continuous deployment The Continuous delivery pipeline ensures stable software ready to be deployed but can be deployed at the desired frequency or with a simple button which you can manually click to start the production deployment but a continuous deployment pipeline
Sahil Puri
Along with everything automates the step of production deployment also so anyhow coming back to the first point of Dimension which is the tools you require latest tools and solution to build these pipelines covering the entire Horizon of your SDLC like testing at each stage and ready integration with third-party tools for example Jira or Slack in your organization I would really recommend you all to try out a POC with some of the front runner cloud provider Solutions like pcloudy which can really help your team in building some efficient CI/CD automation
Sahil Puri
Pipelines and the right test approach okay so as they rightly say the age of automation is going to be the age of do-it-yourself which is very evident from some of the above strategies that we have discussed in the second dimension now if you would note there is no dependency to be created on DevOps or any other team to design our pipelines monitoring the environment setting up our own environment in boxes dashboards analytical reports and we take the responsibility of our own systems as quality engineering team take time to learn and strategize but slowly remove all the dependency on other teams for the above mentioned task and that is what our second dimension is all about so a quick recap in the second dimension
Sahil Puri
We started to focus on entire system the tools the Frameworks design patterns and Reporting which is the core of our automation strategy thus ensuring the success and return on investment by solving the when where how of automation so by now in our first two Dimensions we have covered the what how when and where to run our automation now coming to the final and the third dimension which gives this strategy a 3D Edge which is all about culture people and collaboration
Sahil Puri
Now this Dimension covers the most important question which is the why of automation yes your team should be very clear of the exact purpose of building this automation strategy and project now it can be to reduce the Redundant efforts of Engineers by executing the test cases in an automated manner it can be an early feedback to the Developers for improving the build quality or it can be to improve the overall production quality or to achieve better coverage or even all of these but you have to be clear with your why once you are aware of the purpose it becomes easier to Showcase its ROI and build upon your strategy and it really opens a lot of gates for your overlord overall growth of the team and the project
Sahil Puri
Next come the mindset the mindset of your team and even the entire organization towards quality Automation and testing now the best way to create a mindset is to find like-minded people across teams and take their help in broadcasting your message and your context to set the right tone about quality across the organization and this should be focused to make everyone understand the common goal that everyone is striving for on a daily basis that is to create value with quality
Sahil Puri
Goal can be better explained with the help of sharing your vision and Mission to achieve this goal now a mission statement defines the objectives and how you plan to reach these objectives and on the other hand a vision statement details the Milestones towards that mission for example a short-term vision for every quarter or on a monthly basis and a long-term vision for quality no once you are done with strategizing and designing and even building your CI/CD pipelines you will realize the importance of other continuous strategies that are less discussed which are continuous testing and continuous collaboration
Sahil Puri
Now coming to continuous testing now it is required for your team and entire organization to move towards the ideal agile software development methodology now without this the testing process remains stuck in the past practices even as organization invest considerable time and money into transforming their development processes to Agile and meet tomorrow's business demands again in order to achieve continuous testing you need to take some critical build versus buy decisions and as per the current market competition digitization and Urgent go to market deadlines it really makes sense for most of the teens to choose and buy these of the self Solutions now this is where tool providers like Pcloudy can really help us in ensuring continuous testing and building quality
Sahil Puri
Focused pipelines from design to deployment stage of our SDLC similarly there is another continuous strategy which is continuous collaboration now what do you mean by a continuous collaboration so it is a process where you closely collaborate with all the potential users of your automation in each steps of your SDLC so that your automation strategy so that you ensure that your automation maintenance and Improvement does not remain a role and responsibility for just you and your team in future rather it becomes an organization-wide effort and this organization wide effort and ownership with responsibility cannot be achieved without continuous collaboration
Sahil Puri
Right so you have to work together with all the cross-functional teams and understand their requirement thereby packaging your Automation in a personalized box and sending to each team for solving their use case so here you are trying to solve another team's problem with some with your product or your built-in system so naturally you have to understand their pain Point areas and speak in their language so using the right set of process and prioritizing people interaction along with tools in our strategy we are trying to build the right
Sahil Puri
You can say Automation and quality culture and mindset now this mindset culture collaboration and ownership that we are talking about play a very important role in defining the leadership support or the senior management support to invest in automation now you need to prove the worth of your system and show them the ROI by right marketing strategies using analytical reports Improvement reports with proof points case studies Etc now remember that even the best products in the market like WhatsApp or any other product required timely advertising and marketing efforts to be on the top now having a strong leadership support
Sahil Puri
And right marketing steps as part of your automation strategy will really help you to get the budget for some third-party tools thus saving a lot of your time from building in-house replicas you would require approvals and budgets for hiring and building the right set of talented team with the required talent and skills next after hiring you need to focus on their growth and training of all the hired members now all these points in the third dimension make up for the right culture and mindset across the organization for building the unbreakable layer of Automation and quality
Sahil Puri
For years and years to come now once you build the right culture and collaboration across the teams you are now working towards a sustainable automation project now what do you mean by a sustainable automation project I would say a sustainable project is one which is both maintainable and scalable and everlasting so you have to keep the motivation really high among the entire team by sharing the point of Utopia for your automation Journey so at last I would really press upon the importance of the need to strategize for continuous Improvement by forming a feedback loop using surveys polls regular feedback or 101s with cross team members to know the Improvement point of your automation system and continuously work towards that point of Utopia or the your expectations and remember no one can help you more
Sahil Puri
Than your critics so if you can if you can filter out the constructive feedback of course so like keep all the feedback coming in from across the teams and just keep improving your pipelines processes automation systems scripts and the tooling and again as they rightly say quality is not an act it is a habit which is evident from prioritizing people collaboration mindset and culture in the third dimension of our automation strategy wherein we are trying to create a habit in each member of our organization to think about the quality
Sahil Puri
In the in the right manner okay so a quick recap the third dimension covers The Who and why of our automation strategy who means who are the stakeholders or users of our automation and wine covers the purpose and vision along with the mission of our automation strategy and quality culture so with this we cover all the three dimension of our multi-dimensional test automation strategy the foundation the automation system and tools and the third dimension being all about culture
Sahil Puri
People and mindset please note that each of these Dimension are equally important and basically pushes and balances each other in a circular fashion therefore even a little imbalance or any depriitization of any of these Dimensions will result in a downfall and possible failure of our entire automation strategy so the First Dimension taught us the what the second dimension taught us when where and how and third dimension taught us the why and who of automation and I think that is the holistic view that we were talking about at the start of our webinar
Sahil Puri
No in the last few minutes you all must have realized a feeling of increasing ownership and our strategy Dimensions going on from me to V that is the team or the testing team to entire organization from just testing and automation to Quality culture and mindset now let's discuss the mapping of these Dimensions that we just discussed to different Team hierarchy levels now it is obviously not possible to construct all the three dimension by a single quality engineer or a software
Sahil Puri
Engineer without the support of the next layer that is the leads and Architects similarly the lead and architect cannot carry the entire burden of this strategy on their shoulders without the support of management and Leadership so how can each layer or each hierarchical level contribute their bit to the success of automation systems yes it is now time to map our 3D model with different team hierarchy levels now the First Dimension that is the foundation it is basically the base to be mastered at the level of quality engineer one or two especially if you are in the zero to three years experience of this role this would give you the confidence to understand the application the design test processes efficient test cases and knowing what is lagging in the production by understanding the production incident stability concerns to solve what to
Sahil Puri
Automate now the second dimension I'm sorry yeah now the second dimension it basically covers the automation system and tools which is which is to be mastered at the level of senior or a lead automation engineer to two architect levels this would give you the clarity about tools Frameworks designs CI/CD pipelines continuous testing and continuous delivery to solve when where and how of automation and finally the third dimension where you would be mastering the people culture and mindset you would really need this Dimension when you're trying for the role of test managers or directors and even higher leadership levels this would really help you to build a sustainable system with the right mindset Clarity of vision and mission across the entire team
Sahil Puri
And high motivation levels across all the members with us with A continuous focus on Improvement and building an entire system finally this in turn defines a three-dimensional growth path and stepping stone for a test engineer career or a quality engineer career so as I told you that this represents a three-dimensional growth path that is the role and responsibility for a quality engineer role try to master the basics and fundamentals first then Master the tools Frameworks Pipeline and the entire system finally to learn how to collaborate and change the mindset about quality
Sahil Puri
Yeah the core or I would say the major learning of this discussion is to rise above testing and start talking about quality to set the right context and the right mindset enable the cross-functional teams to own quality as we say quality is everyone's ownership but we are the enablers so enable the cross-functional teams to own this quality with different quality Gates present at different stages of your software development life cycle and you will see yourself turning into a great quality engineering leader very soon so on this note I would like to end today's webinar with a question for all of us that is there a fourth dimension
Sahil Puri
That you have experienced in your journey and would like to discuss then please come up with your points in the following Q&A sessions and thank you for being such a lovely audience you can reach out to me on LinkedIn to discuss more on this topic generally over to you thank you sahil for an enlighting and interesting sessions so it's time for Q&A now the audience if you have any questions you can ask away you can drop your questions on the Q any box that you see at the bottom of the screen
Sahil Puri
Here's a question from pratik he is asking do we have any solution for car Player on Android auto hey hi pratik yes that's surely an out of context discussion but first of all I'm not very sure of what solution are you asking is it a testing solution or an automation solution because yes surely there are both testing and automation Solutions are very available for carplay or Android auto again provided if you are trying to test just the software
Sahil Puri
Or the entire firmware or even the hardware so your approach would matter and I would really suggest you if you can break out these layers test the hardware in isolation test the software in isolation and then combine into system testing that would be the best way to test these kind of systems like Android auto like is I see a question from Manish Connecticut yeah so Manish is asking what are your thoughts on in Sprint Automation in agile and how should we handle it so Manish I would say it's it's a journey you start with clearing the backlog of your regression and Sanity and smoke test and build those pipelines and
Sahil Puri
Train the team with the right set of trainings or tools and when you see yourself invested for about three to five months and your backlog clearing to around 70 to 80 percent then I think it is the right time and it kind of becomes inevitable in order to maintain the current scripts that you switch to in Sprint automation or maybe plus one Sprint automation but yes I would say I definitely support the idea of in Sprint automation at least try and cover the smoke or the sanity part while doing while working on a feature ticket before its deployment that would really help to keep our scripts updated maintained and our automation coverage
Sahil Puri
On the top levels okay ravindra is asking testers are remembered when there is an issue how to change that mindset so ravindra I think we discussed that point in very detail and I definitely empathize with you but that is the third dimension all about is creating the mindset for quality and I touch this topic also to how to create how to enable different teams and different departments to ensure quality at their level so you can play the role of that enabler so for example if there is an issue and you are remembered so first of all that's a good thing go on and understand the RCA understand why you miss that bug and what all you can do to save that leakage but
Sahil Puri
Then go above the ladder try to do the bottom-up approach and understand why it was missed from the unit test on the integration test from the developer end why it was missed from the code reviews and even why it was introduced in the first place so I think we have to live through the Journey of finding books to preventing bugs and once you start understanding this difference on how to work towards preventing bugs by collaborating with your development product teams I think your problem will start to solve slowly yeah so pranup is asking for a QA manager strategy he joins an organization and how to change modify and involve the framework how to go about tweaking the culture tool adoption so the first mistake that most of us makes is trying to come with
Sahil Puri
Our own baggage of experience and try to change the culture so the first idea is not we don't have to change the culture the first idea is as a manager when you join a new organization you have to understand in the current culture the current set of tools and framework and then try to make relations to understand the as I told the good part the good points and the pain points there must be some pain points and there must be some loopholes or Improvement points that are still left because that is true for any system try to pick those Improvement points and bring your experience into the picture with improving the current system only rather than changing the entire game from 180 degree that would not solve the purpose
Sahil Puri
How ROI is calculated for an automation plan I think this is an anonymous question so I think as I again touch the point we have to be very clear of why when deciding or strategizing the automation plan so the ROI would come from your why only why on the first place you decided to start automation now was it to achieve the right set of device coverage was it to achieve the right set of code coverage or even was it to reduce some of the possible efforts of your own qat members by automating the checks right and when your why you can collect some data Maybe on a monthly basis to check the ROI and what gain are you exactly getting on those whys which you decided much before right so it again come boils down to your strategy which
Sahil Puri
Has to be perfect yeah I think the last question which says that how do we convince management who is bit reluctant to change their mindset about tool selection I think I would say just one word collaborate relation build relations okay make them understand your experience and your pain points if you are using that tool and not finding it Worthy and again most of the logical discussions end with data and research so try out proof of Concepts try out different tools available in available in the market and Mark them against a set of categories like cost like availability like service customer
Sahil Puri
Service after sales any kind of network lag or flakiness and when you give these tools some points against these four or five categories and the tool that wins will be a no-brainer to choose so that will avoid any kind of a pre-bias for any particular tool asking other teams and majority seniors always say why testers take so long to test without understanding the details and how you deal with that ravindra the right answer is you have to learn from your experiences and give again give data to prove that you would require X number of days to test a particular feature you can complete that
Sahil Puri
In one day also or in two day also x minus two days but then what is the risk that is getting left in those two days that they are trying to shrink and once you tell them the risk it is again a business product Tech and QA so I would say it's a joint discussion and decision to again a trade-off between speed and quality and the risk so our job or your job is to set out the right risk and bring them on the table and discuss with everyone so vikrant is asking do we need to include business validations in the automated regression test Suite
Sahil Puri
Dependent modules I think Vikram this question is very much specific to your use case normally yes we do cover the business validations in smoke or even sanity or regression test suites because those are the P0 requirement as part of first traceability Matrix but if your pipeline or the it is taking too much time then you can again discuss with the business guys and the product guys and see what do what would they prefer and take a collaborative decision automation solutions for UI based Windows application like sqli image Horizon since we are trying POC on saturation we Face frequent issues so again you're trying to build some
Sahil Puri
In-house replicas which are already present in the form of some latest Cloud Solutions including Pcloudy so I would suggest to get in touch with these tools and the tool providers have a demo have a POC and even try to work very closely with the tool POC to create a hypothesis for your management when you are living through this demo and the phase of creating hypothesis and POC you will see yourself learning a lot during this time and hopefully you get the approval and you solve the problem with the available tools rather than building in our Solutions
Jeroline
Okay thank you sahil for answering these questions and for the great presentation so it was a pleasure to have you with us and I'm confident that the strategies and insights that you shared will be of great help to all of us who work on improving our test automation efforts and strategies so with this we'll conclude the webinar thank you all for attending and we hope you have learned and enjoyed this presentation and remember to keep breaking free and pushing the boundaries of what's possible in the QA industry so see you at the next breakthrough session until then bye thank you bye
