Beth
Hello and welcome to today's web seminar move beyond traditional mobile app testing with AI and DevOps sponsored by Pcloudy and featuring one of
Beth
The company's co-founders as our speaker Avinash Tiwari I'm your host and moderator Beth ramnik thanks for joining us today before I hand it over
Beth
To our speaker let me explain the console you see in front of you the three panels on your screen can all be moved around
Beth
And resized at the bottom of your console is a widget toolbar by clicking the handouts widget in the middle the one that looks like
Beth
A piece of paper you can download a PDF of the speaker's slides if you have any technical issues please let us know by typing
Beth
In the questions and answers panel to the left this is also how you can ask questions there will be a Q&A session at the
Beth
End so feel free to ask questions throughout the presentation and finally to optimize your bandwidth for the best viewing experience today please close any
Beth
Unnecessary applications you may have running in the background this web seminar is being recorded and it will be made available to watch on demand
Beth
When the recording is ready to watch you'll get an email with instructions about how you can access the presentation and with that I'd like
Avinash Tiwari
To hand it over to our speaker Avinash Tiwari thanks bit thanks for setting it up and good morning or good afternoon or good evening
Avinash Tiwari
Depending on which part of world you are in it's a real pleasure to be part of group of experts and a global audience thanks
Avinash Tiwari
A lot for joining in so let me begin with a quick intro about myself my name is Avinash and I am a co-founder of
Avinash Tiwari
Pete Laurie comm one of the fastest growing mobile app testing platforms right now
Avinash Tiwari
So with this big brief introduction I hope you all are settle and ready to go and excited to take you through the webinar today
Avinash Tiwari
I like solving problems of testing themes and the created Pcloudy calm with a goal to simplify lives to be male more precise testing
Avinash Tiwari
Lives of app developers and testers and in this journey we have made some exciting progress and that's what I'm going to show you today
Avinash Tiwari
So in that context I am going to focus on three key areas the first digital transformation and how it's impacting the world of mobile
Avinash Tiwari
Apps the second one is focused around quality at speed and what can we do to take it beyond what you have already seen and
Avinash Tiwari
Lastly what's the surprise with Pcloudy and how Pcloudy helps you take it beyond and this is where I'm going to show you
Avinash Tiwari
A small demo as well towards the end of the webinar so over the next 35 to 40 minutes I'm going to cover these three
Avinash Tiwari
Topics and towards the end of it we'll take the Q&A so please feel free to drop in as many questions as you can and
Avinash Tiwari
Share your opinions so are we not living in an amazing time of Technology why is per se the world around us is becoming all
Avinash Tiwari
Digital with limitless possibilities today as a consumer you can pre-order your coffee at the same time interact with augmented reality in the store and
Avinash Tiwari
When it comes to the payment you can skip the lines with alternative payment methods to cut the story short as a consumer we have
Avinash Tiwari
Access to unlimited goods and services and all this is happening because of the mobile technology which is around us so Moodle is becoming the
Avinash Tiwari
Center of all this change it's the tool which is the which is bridging the gap between digital and physical
Avinash Tiwari
And the proof is explosion of the app world so in the first quarter of this year Apple had close to 2 million apps on
Avinash Tiwari
App Store and around similar time Android users were able to choose between 3.8 million apps it's a clear indication that more and more businesses
Avinash Tiwari
Are adopting mobile apps as one of the primary channel for next business growth and in fact a lot of new businesses are either going
Avinash Tiwari
Mobile only or mobile first and that's the reason of this explosion in the world of apps
Avinash Tiwari
So as a business mobile apps are not just another channel and more and more organizations are realizing that it can be a mean to
Avinash Tiwari
Create awesome customer experiences and there are numerous examples around us and one of the example Starbucks I am sure you all are quite familiar
Avinash Tiwari
With the secret of Starbucks app success is it is the fact that it gives intuitive experience to the user so it makes the store
Avinash Tiwari
Discovery very very simple you can order your order your coffee through the app and make the payment through the wallet app that they provide
Avinash Tiwari
By just showing it across the barcode device that they have and recently Starbucks also has taken the mobile app experience to the next level
Avinash Tiwari
By bringing in conversational ordering systems conversational ordering system powered by AI so Mobile is becoming all about customer experience and it's not just about
Avinash Tiwari
Experience only when it comes to mobile it's all about speed so how quickly you can make a change and let the customer experience it
Avinash Tiwari
So the mathematics of velocity matters a lot when it comes to mobile apps
Avinash Tiwari
And this demand for market speed has left companies with a complex problem how do we go faster than ever before without losing an eye
Avinash Tiwari
On quality and there are quite a number of success stories like Facebook and Netflix so the Facebook mobile app is updated and refreshed every
Avinash Tiwari
Two weeks like clockwork and that's the new normal we'll talk about it we'll talk about this example in detail a little bit later in
Avinash Tiwari
The webinar but just to set the context about what speed means in the mobile apps
Avinash Tiwari
So I call this intersection of experience and experience and speed as quality at speed and it's a no-brainer for me to say that quality
Avinash Tiwari
At speed can be achieved with two fundamental principles when it comes to software development practices the first is a child which I'm sure you
Avinash Tiwari
All are aware of it allows team two it allows teams to work closely with business and it pulls quality forward as foresters say says
Avinash Tiwari
It and it's also called shift left of quality so you are delivering small chunks to end customers on weekly or monthly basis unlike classical
Avinash Tiwari
Models where the releases where either end of a quarter or six month or a year so this agile practice of dividing deliveries into chunks
Avinash Tiwari
Allows teams to get feedback early on from the customers the second principle which brings speed is DevOps and it's a sift left of operations
Avinash Tiwari
So dev of DevOps practices basically allow you to deploy the software and the environment on it on which it runs automatically and on demand
Avinash Tiwari
Across the software delivery lifecycle so you can truly have multiple deployments in a day and lot of organizations have already successfully implemented and achieved
Avinash Tiwari
This state so that's what is a kind of speed market is looking at
Avinash Tiwari
Then when we talk about DevOps and continuing this DevOps discussion a little further we all know that DevOps is enabled by two practices continuous
Avinash Tiwari
Integration and continuous delivery but will these practices work in sync if you don't have continuous testing process in place so what you see here
Avinash Tiwari
In this diagram are the three three principles of DevOps working together like a here if any one of the gear fails or component within
Avinash Tiwari
The gear fails the system breaks down so continuous testing becomes an extremely important part of overall tab ops churning and that is what is
Avinash Tiwari
Of our interest when we when we talk about Pcloudy as a product and I am sure you all are part of testing team
Avinash Tiwari
Or development team making sure that the continuous testing practices are followed so
Avinash Tiwari
Let's validate our thoughts through some independent studies and surveys so here what you see is the data extracted from world quality report 2017-18 showcasing
Avinash Tiwari
The popular and best practices followed in DevOps and there are two highlights that I have done in the slide here and if you if
Avinash Tiwari
You look at the data closely or the first highlight
Avinash Tiwari
Clearly shows that 87 percent of CIOs and senior tech professionals are either using or planning to use a cloud base test environment and depending
Avinash Tiwari
On the choice of organization it could be a private or a public cloud or on moon cloud but it has become a must to
Avinash Tiwari
Have to achieve speed in the DevOps journey and the second data point here which is highlighted is about continuous testing which we were just
Avinash Tiwari
Talking about and that also clearly shows that 80% of respondents have chosen this as a preferred DevOps practices so in the context of testing
Avinash Tiwari
Continuous testing and use of a cloud-based test environment are two key findings when it comes to implementing DevOps successfully and we are going to
Avinash Tiwari
Kind of elaborate on both of these as we discuss further
Avinash Tiwari
So you we validated this with some surveys and report but what are some of the successful stories saying about it and one of the
Avinash Tiwari
Big examples are one of the most successful examples come from the Facebook journey in terms of their product releases and the way they ship
Avinash Tiwari
The product quite often and they call it ship early and ship of an approach so with this approach they have been able to update
Avinash Tiwari
Their web app multiple times in a day while their mobile apps are updated at a frequency of almost two weeks and they can do
Avinash Tiwari
It because of the DevOps pride practices that they have adopted and they adopted it quite early on in their product journey and here I
Avinash Tiwari
Would like to emphasize about one of the key takeaway from this success story especially for mobile teams and the takeaway is about their massive
Avinash Tiwari
Investment in the device infrastructure so every time their developers make a change it it gets tested on 2,000 real devices which they have set
Avinash Tiwari
Up in their data center like on Prem cloud of devices
Avinash Tiwari
So with this approach they have been able to bypass use of simulators and emulators because real devices are the devices which are going to
Avinash Tiwari
Be used by end-users so why do we need to why should someone test it on a simulator first and then do it on real
Avinash Tiwari
Devices but of course it's difficult for every organization to have this massive infrastructure but that's where companies like Pcloudy your product like peak
Avinash Tiwari
Lorry comes into play and this this is also in sync with the survey report that we saw the importance of cloud-based test environment which
Avinash Tiwari
Came across prominently in the data moving on at this point we are quite sure about role of DevOps in bringing up in bringing speed
Avinash Tiwari
To the delivery but the question is is mobile app dev ops different than the practices followed for other kind of application like web apps
Avinash Tiwari
Or other client-server applications so the answer is fundamentally no the practices are almost same but mobile DevOps definitely poses some unique challenges and it's
Avinash Tiwari
Very important to understand those challenges before you start thinking about mobile dev ops some of the key ones I have highlighted here and I
Avinash Tiwari
Will talk about those briefly the first one is the multi-platform support so mobile apps do not have a single environmental target right most mobile
Avinash Tiwari
Apps target multiple devices multiple operating systems which means dealing with device fragmentation and this poses a challenge for ops team to create an optimal
Avinash Tiwari
Device infrastructure and we saw that in the example of Facebook as well they had to create a massive infrastructure to cover the the end
Avinash Tiwari
Device target that their customer segment a customer segment is going to use and moreover the device needs are ever-changing
Avinash Tiwari
So that that poses a unique challenge when it comes to mobile the second challenge that I want to highlight here is
Avinash Tiwari
Around enterprise mobile apps especially enterprise business to customer or business to employee typically these kind of applications have very less business logic on the
Avinash Tiwari
Mobile device itself instead these b2c or b2e mobile applications primarily serve as a front-end to the enterprise applications already in use by the companies
Avinash Tiwari
Like order management systems or HR systems or sales management systems so this I mean what what it means that mobile apps need to be
Avinash Tiwari
Developed and delivered in conjunction with the back-end systems so for DevOps the challenge is to think holistically of all the application in the enterprise
Avinash Tiwari
And coordinate their build and release processes and cycles so the mobile team might not work in a silo as far as the
Avinash Tiwari
Enterprise mobile apps are concerned and the third challenge is because of the way apps are distributed and here it's termed as app store so
Avinash Tiwari
The app store adds an additional or synchronous step to the deployment process because developers are unable to deploy app updates on demand so even
Avinash Tiwari
For a critical bug fix the developers will have to go through an app store submission and review process so continuous delivery becomes submit and
Avinash Tiwari
Wait and the last challenge which I want to highlight here is around the build process of mobile applications and this is a unique challenge
Avinash Tiwari
And again it requires a massive investment in infrastructure so it's linked again to the multi platform support of applications where you require several builds
Avinash Tiwari
Of the applications to be created each time the application goes through a change so for example for application you need a apk for iOS
Avinash Tiwari
Application you need IPA and again for iOS applications if you have to support multiple versions like 12.0 or the lower versions like 10.0 or
Avinash Tiwari
9.3 the xcode versions change and you require to build with those respective X Xcode versions to generate the apt IPAs so the build setup
Avinash Tiwari
And configuration for each supported mobile environment is different from others so you are most likely needed to create a small form of build servers
Avinash Tiwari
Which can be used by the development teams to seamlessly create their builds whenever there is a change and it's a huge infrastructure issue so
Avinash Tiwari
Either you need to think about creating it in-house or there are cloud systems which can be used so these are some of the key
Avinash Tiwari
Challenges which makes mobile dev ops quite unique so so at this stage we have reached a point where we can look at a typical
Avinash Tiwari
Mobile interwebs cycle so diagram that you are going to see now is a simpler diagram than what you are going to implement in reality
Avinash Tiwari
But it will give you a sense of what are the key components when it comes to mobile devolves and how the process flows
Avinash Tiwari
So let's try to depict the mobile dev app cycle through a diagram
Avinash Tiwari
So I have divided into two different stages however in a DevOps term there is nothing like stage but for simplification for simplicity of explaining
Avinash Tiwari
I have taken that liberty to term it into two different stages so what you see here right now I am calling it dev stage
Avinash Tiwari
And in depth stage the the developers are while developing the code they require access of real devices or simulators to debug in real time
Avinash Tiwari
But the real process starts when they commit the code to a versioning system like github and it's where the continuous integration server triggers the
Avinash Tiwari
Build creation and this is where the build server comes into play which we were talking about so these build servers either will be an
Avinash Tiwari
Internal build form or external system like circle CI and there are lot many other systems which can be used as soon as the build
Avinash Tiwari
Is completed automated unit tests are run and typically these unit tests are either espresso scripts or X UI test scripts written mostly at the
Avinash Tiwari
Code level or the or to test the services or the api's
Avinash Tiwari
Once the build is successful it moves to the second stage again this stage definition is mine not a standard definition which I am calling
Avinash Tiwari
It a QA stage so if the unit test is passed the automated functional regression tests are triggered and
Avinash Tiwari
That again has to be run on a real device infrastructure or emulator simulators depending on the choice of the organization and once those tests
Avinash Tiwari
Are passed then if needed manual app compatibility tests are done and then the app either goes back to the developer depending on whether bugs
Avinash Tiwari
Are present or not and the same cycle continues and if all the cycle all the steps are passed the app is pushed to either
Avinash Tiwari
Pre prod or for App Store submission so this kind of overall gives you a sense of what a mobile dev ops journey can be
Avinash Tiwari
And the yellow circle that you see here kind of highlights where you need a real device infrastructure as an organization we have seen a
Avinash Tiwari
Benefit of using real device infrastructure right from the beginning itself but a lot of organizations choose to do do to work on emulators during
Avinash Tiwari
The development stage and they use real devices during the testing stage but our recommendation is to use real devices as much as possible
Avinash Tiwari
But one key thing that you you must have notices the different testing stages in this mobile develops journey and that is this is what
Avinash Tiwari
The continuous testing is all about so every stage has a testing phase attached and until the testing is passed it cannot move to the
Avinash Tiwari
Next stage so this is where a continuous testing come phrase comes into play and if you see this the success of this seamless process
Avinash Tiwari
Depends on how much automation you have done so it's all about automating the tests as much as possible to get the real benefit of
Avinash Tiwari
DevOps and to really use continuous testing effectively great so
Avinash Tiwari
We have seen the journey so far we talked about DevOps practices use of continuous testing and it's been quite a journey for organizations to
Avinash Tiwari
Move from classical testing to agile and then DevOps but what next
Avinash Tiwari
Where is the future heading towards the DevOps practices has already made the the already improved this speed of delivery but can we speed it
Avinash Tiwari
Further is the question and that is where AI comes into play so here we are using a term called digital testing where there are
Avinash Tiwari
Lot more components coming into play as far as applications are concerned so mobile apps are getting connected to external devices like variables mobile apps
Avinash Tiwari
In itself has AI components like chatbots these have machine learning capabilities to predict customer behavior and then create workflows so all these are in
Avinash Tiwari
The system more complex and creating a demand of further speed into the development process so that's where the digital testing comes into play and
Avinash Tiwari
We see that AI can play a big role in next stage of journey in terms of improving the speed of delivery so this is
Avinash Tiwari
What we are calling it going beyond continuous testing where more and more use of AI based system will be done in the testing phase
Avinash Tiwari
And what does it mean there is an ongoing debate happening in terms of what AI can do especially in the context of mobile app
Avinash Tiwari
Testing and that is where our interest lies and there has been lot of I mean there are a lot of experts who are saying
Avinash Tiwari
That AI has the ability to replace testing completely I am NOT an Oracle but I don't see this happening in near future but we
Avinash Tiwari
Are certainly very excited about capabilities that AI present in front of us and as an organization when we started thinking about implementing AI within
Avinash Tiwari
Our platform we thought on three different lines to solve some of the testing problems of mobile teams and the first was can it improve
Avinash Tiwari
The speed of current testing process so the question was around can AI be used to create a personal assistant for testing so we have
Avinash Tiwari
Been using personal assistant for our daily use like Siri or Alexa but can we create a personal assistant for testing which can become buddy
Avinash Tiwari
Of mobile teams and then give them a power of AI to to further increase their speed the second aspect that we looked at was
Avinash Tiwari
Can we generate meaningful data for users to make intelligent decisions so something like based on the historical data of bugs or logs can we
Avinash Tiwari
Predict what kind of failure can happen with a new change in the application so that the testing can become more focused on the failure
Avinash Tiwari
Areas rather than going for full testing without intelligent decision and the third aspect was can it improve the test coverage and reduce cost so
Avinash Tiwari
There is a lot of focus right now on exploratory testing for mobile as well as web because that is where a lot of these
Avinash Tiwari
Corner scenarios crash scenarios are found out when the the application goes into the market so can e I be helpful in exploratory testing so
Avinash Tiwari
That increase the test coverage and find out those kind of corner issues before those issues are found out in the production so these are
Avinash Tiwari
The three different areas that we looked at and we started integrating the AI capabilities in our platform so the question is are you really
Avinash Tiwari
Ready to adopt a personal assistant for testing are you ready to say certifier can you help me test my app that's the question I
Avinash Tiwari
Am posing in front of all of you some of you might have used certain BOTS
Avinash Tiwari
But if you have not are you ready to use these kind of systems
Avinash Tiwari
But before I talk more about certifier I think I would like to take a quick pause because it's the right time to take your
Avinash Tiwari
Opinion we have talked about mobile we have talked about DevOps we have talked about use of AI so it's important that we understand what
Avinash Tiwari
Exactly you have been using so far and it will really help us as well in understanding the customer situations and their needs so we
Avinash Tiwari
Can take a quick poll it will take maybe 15 seconds for you to interview options so the poll is about what practices your test
Avinash Tiwari
Teams are using to keep pace with digital transformation especially in the context of mobile app development and testing and here the options are are
Avinash Tiwari
You already practicing continuous testing how much of automation test runs are are you doing so do you have automation runs across
Avinash Tiwari
For for each build and code change are you already using cloud-based test environment and the last one is
Avinash Tiwari
Have you already started using AI based platforms to increase your testing speed
Avinash Tiwari
So I can already see the responses coming in so I'll just wait for a
Avinash Tiwari
Few more seconds before we move forward but thanks a lot for participating in the poll I
Avinash Tiwari
Can see the as as it's clear from our other [Music] surveys that we talked about continuous testing definitely is one area where a lot
Avinash Tiwari
Of organizations have already adopted it automation and continuous testing is all about automation so there are quite a number of responses around automation however
Avinash Tiwari
Cloud based test environment the result here is slightly surprising to me I wanted to see from my experience it should be more
Avinash Tiwari
But very less use of AI so far and it's quite understandable so I'm going to the next
Avinash Tiwari
Slide where the result is published here so it's quite in line with what we have seen so most of the organizations using DevOps have
Avinash Tiwari
Already implemented continuous testing use of automation however as I said cloud-based test environment usage is slightly less than expected here in the probably community
Avinash Tiwari
That you're talking to here and the AI is very less but I'm sure what you see now is going to create interest for you
Avinash Tiwari
And you can start looking at it clearly so we are all set to see what exactly Pcloudy offers in terms of the DevOps
Avinash Tiwari
And how AI is being used within Pcloudy as a platform and how it's helping customers use it effectively in their DevOps journey but
Avinash Tiwari
Before that for some of you who is some of you who are not very familiar with Pcloudy just a quick introduction it's an
Avinash Tiwari
Enterprise mobile app testing platform in terms of key features it provides real-time testing on real devices so you can use it the way you
Avinash Tiwari
Use a physical device for testing and the platform has one of the fastest speed when it comes to access of the real devices it
Avinash Tiwari
Enables continuous testing so starting from automation creation to execution on parallel devices it enables organization to perform their continuous testing both at the unit
Avinash Tiwari
Test level as well as functional test level and lastly in terms of the key feature AI base testing the our platform comes in to
Avinash Tiwari
Multiple flavors as mentioned in the slide here one of the key flavor is public cloud where we have more than 500 plus devices hosted
Avinash Tiwari
On our cloud across various locations in the world but a lot of Enterprise a--'s are using private or on-prem cloud because of security reasons
Avinash Tiwari
Because of the scalability that they want to look at so Facebook example that we talked about was on prim cloud but they have developed
Avinash Tiwari
It on their own but we have the flavor where we can help you set up on frame or a private cloud to help you
Avinash Tiwari
Achieve your mobile divorce journey so with that introduction a quick snapshot of where does peak laude fit into mobile dev ops journey so this
Avinash Tiwari
Is the diagram that we saw earlier and I'm going to highlight where exactly peak laude fits in so when developers are looking at real
Avinash Tiwari
Device real devices for debugging we have a feature called device tunnel which enables developers to access the real devices directly within their development environment
Avinash Tiwari
So it could be XCore or Android studio or eclipse or tools like cross-platform development tools I examine they can access the device directly from
Avinash Tiwari
Within their ID and then do a real time real time debugging as part of peak laude we support execution of unit tests on real
Avinash Tiwari
Devices using tools like XE test or espresso and we have a we have one of the most exhaustive list of tools supported for automated
Avinash Tiwari
Functional tests so appium calabash robotium and we have our own automation studio called aapki where you can create your scripts as well as execute
Avinash Tiwari
The scripts and as I said the platform has extensive number of features to allow app compatibility testing on real devices and the last one
Avinash Tiwari
Is synthetic monitoring on production so that again can be used with be cloudy as a platform
Avinash Tiwari
So now we are all set to see things in action whatever we have talked about I would like to quickly showcase what we have
Avinash Tiwari
In our kitty and how it can improve your mobility for Chevy so I am going to switch to my desktop screen sharing so just
Avinash Tiwari
Give me a moment quick so I am logging into our public cloud here so I'm just using my credential to log in here
Avinash Tiwari
So violet logins I am going to quickly showcase you some key use cases before we move to the certifier part of it where I
Avinash Tiwari
Want to showcase how certify a bot can help you and can become your buddy for testing so I've logged into the public cloud platform
Avinash Tiwari
Here and I'm going to the device tab where you can actually see a host of devices which are available on our public cloud but
Avinash Tiwari
The same look and feel is there for a private cloud or on-premise setup as well I'm not going to detail off the features I
Avinash Tiwari
Just want to quickly show you one simple use case so I'm connecting to a device which is Google pixel and it takes couple of
Avinash Tiwari
Second to connect so the device is connected here and I can access the device in real time and if I just click on install
Avinash Tiwari
Here I can quickly install an application while the application is installed I mean I can take screenshots any number of screenshots
Avinash Tiwari
If I have if I have to log a bug based on the screenshot I can just log it directly to JIRA or slack and
Avinash Tiwari
At the same time I can monitor logs in real time I can monitor performance of the application in real time
Avinash Tiwari
So you can see sheer data like memory CPU network usage battery usage getting plotted in real time and there are several features with respect
Avinash Tiwari
To app compatibility testing on a real device things like geolocation network simulation I'm not going to get into that detail right now so for
Avinash Tiwari
The time being I'll release this device I just wanted to give you a sense of how the cloud looks and how it can be
Avinash Tiwari
Used for a quick use case of app compatibility so just release the device I talked about automation and
Avinash Tiwari
How we are supporting one of the largest tool sets so I talked about aapki which is our automation studio using which you can create
Avinash Tiwari
Scripts and run it on devices in parallel apart from that we support other tools like monkey espresso calabash opium and xui test as well
Avinash Tiwari
But let's go to the key part of our so case which I wanted to show today which is certified the bot I was talking
Avinash Tiwari
About so are you really ready to use BOTS as your buddy for testing so this is what we certify our interface looks like and
Avinash Tiwari
This is typically used this interface is typically used by the QA teams along with the app compatibility test that they do so it's a
Avinash Tiwari
Very simple interface where you just need to select your app which is already uploaded on the cloud here so I just selected a sample
Avinash Tiwari
Demo application and I can provide a session name and then I can provide certain test data if the application requires so if you require
Avinash Tiwari
Some user login credential I can pass it I can add more data sets if the application needs however the bot will add random data
Avinash Tiwari
While it navigates to the application so for example if it means mobile number I can pass it on and once I'm done with my
Avinash Tiwari
Test data configuration I can just say run and that's it the bot will self configure all the tests for this application and start executing
Avinash Tiwari
It on multiple devices in parallel so I am just going to show you a sample report Here I am NOT going to wait for
Avinash Tiwari
This test to get completed so this is one of the report which has been generated to certify so what you see here is a
Avinash Tiwari
Dashboard interface and you can see some data points like rating for the application in terms of its CPU performance its memory performance its battery
Avinash Tiwari
Network and this is an average data based on all the devices on which the test has been executed but you can look at device
Avinash Tiwari
Level data as well so this test was executed on 10 devices in parallel and you can see the bot has chosen different versions to
Avinash Tiwari
Give you optimal device coverage for this testing the bot has its intelligence to pick the device coverage on its own however we are coming
Avinash Tiwari
Up with a feature where you can select your choice of devices as well so you can see the data at the device level in
Avinash Tiwari
Terms of its performance and then what generates certain insights depending on the logs and the reports it analyzes so that's at the dashboard level
Avinash Tiwari
But what will be interesting is to see slightly more detail so there are two kind of tests which are done by a bot the
Avinash Tiwari
First is the crash test where it it is like a smart monkey there it generates random events across the application when it's navigating through
Avinash Tiwari
The flow so this crash test has been done on you can see there are several devices on which this test has been done and
Avinash Tiwari
You can see the top level status of the different activities so installation is past crash test is past uninstallation is past so in a
Avinash Tiwari
Very short period of time you can do a test a crash test for your application and then it generates extensive set of data around
Avinash Tiwari
The performance while it's navigating through the application and all this data is present here for users to analyze and if there are any crash
Avinash Tiwari
So there is a crash here in this application which has been captured in form of crash test dot txt this is the log for
Avinash Tiwari
The crash which has been captured here but let's go and see one of the exploratory bot tests where it tries to mimic the end-user
Avinash Tiwari
Behavior so again the exploratory bot test has been run on multiple devices here and as you can see
Avinash Tiwari
This also has similar format in terms of data capturing so it captures all the data when the test was getting executed like battery memory
Avinash Tiwari
CPU network frame rendering and then it generates all the snapshots so the snapshot will give you a picture of what flow was executed by
Avinash Tiwari
The bot however we are also trying to map it in terms of scenario was right now you can go through the snapshots to see
Avinash Tiwari
What flow was executed and then again you have the logs to analyze if you feel there is something wrong during the execution of the
Avinash Tiwari
Flow so this is the way both crash tests and exploratory bot test gets executed and without any manual intervention just schedule the run and
Avinash Tiwari
All the tests are self configured and executed by BOTS
Avinash Tiwari
But the the question is how do we use it in a DevOps scenario can can it be can it be used by developers as
Avinash Tiwari
Soon as they do the code check in so while they run the unit test which is more at the EPI or service level they
Avinash Tiwari
Get a quick feedback on the functionality of the application as well and that's where the bot test comes pretty handy and that is that
Avinash Tiwari
Can be very easily achieved with me clouding so we have specifically developed a Jenkins plugin for our bot which I'm going to demonstrate here
Avinash Tiwari
So what you see here is the jenkem interface where we have already created a project the project name is Android build test be cloudy
Avinash Tiwari
And if I just show you what's the configuration of this project it will give you a clear idea as to how it works so
Avinash Tiwari
I click on configuration here and as you can see
Avinash Tiwari
We are picking up the we are picking up we are providing the repository URL here with the credential so that the as soon as
Avinash Tiwari
The developer checks in the code will be picked up from that repository and
Avinash Tiwari
You can also specify which branch to use of course a simple application so there is only one branch but you can add multiple branches
Avinash Tiwari
Depending on your application configuration and here since it's since it's an Android application we are using a build system build server called Gradle
Avinash Tiwari
But you can choose to use other build service as well that's provided by Jenkins itself
Avinash Tiwari
So we just provided the wrapper location and the task name and this is where the peak laude certifier configuration comes into play so you
Avinash Tiwari
Have to provide username and API key followed by the CLI and the app path and the CLI here is a simple jar utility which
Avinash Tiwari
Will execute the bot test depending on the configuration which has been provided here and the app path is the jenkins workspace part where the
Avinash Tiwari
Build will be storing the application once the build is completed and here we have also integrated with slack so that the result can be
Avinash Tiwari
Posted in to slack channels or it can be done with JIRA as well so once this configuration is complete the certify a bot will
Avinash Tiwari
Be executed as soon as any code change is done and right now we have been able to provide the result within 10 minutes of
Avinash Tiwari
Execution which we are trying to optimize further but within 10 minutes you can get a result on exhaustive set of devices it could be
Avinash Tiwari
10 it could be 15 or 20 so here we are executing the build manually but in a real situation in build will be trigger
Avinash Tiwari
Automatically and you can go to console output and start seeing the result of the execution so you can see here first it's trying to
Avinash Tiwari
Build the application from the fit repository and once that is done it starts executing the certifier session so you can see here the message
Avinash Tiwari
You have successfully scheduled the certifier session and once that is done
Avinash Tiwari
Once that is completed it will take some time the report is also posted back in the console so this report link is public which
Avinash Tiwari
Can be passed to anyone so you don't need to have a credential to access the login and you can configure the URL to be
Avinash Tiwari
Made public so there is security in place and as you can see the URL has been posted to slack as well within the channel
Avinash Tiwari
That you want to be posted so as soon as you are a certified build is complete the report will be available both in form
Avinash Tiwari
Of email notification as well as slack or JIRA or within the Jenkins console as well so within a very short period of times and
Avinash Tiwari
This is the same report that I showed you earlier as part of the manual certifier session where you'll get a dashboard where you can
Avinash Tiwari
Compare the performance of the application along with the functionality across grass and exploratory pot tests so we have seen a lot of organizations started
Avinash Tiwari
Using it during their QA stages as well as within their development stages because one of the key challenge that we have seen is developers
Avinash Tiwari
Do not have a view of how the functionality of the application is when they make a change the unit tests does not give them
Avinash Tiwari
Full picture of what the functionality of the application is so while certifying and give them a sense of whether at the functional level the
Avinash Tiwari
Application was passed was there any trash and information like that so with this I would end the demo and go back to the slide
Avinash Tiwari
Just to conclude so just wanted to end before we open it for a question-and-answer
Avinash Tiwari
So we talked about a digital age which is forcing us to test smarter and continuous testing is definitely the first step and it came
Avinash Tiwari
Across the survey also that most of you are already following the continuous testing practices but effective use of AI can definitely take it to
Avinash Tiwari
The next level and aie has to be treated as augmentation of the capability and it can really help capabilities of the existing team and
Avinash Tiwari
They could take more intelligent decisions and the last part is peak Laurie has been at the forefront of innovation when it comes to mobile
Avinash Tiwari
App testing and we are committed to keep you ahead in the market so with these thoughts I would end the session thanks a lot
Avinash Tiwari
For joining in and listening I can see a lot of questions
Avinash Tiwari
So we are open to Q&A and feel free to drop in all the questions and you can always reach out to me on these
Beth
Email ids which are mentioned if you need any more information around product or any questions that you have great Thank You Avinash so as
Beth
He said we've already got some great questions here but feel free to continue asking questions and we'll get through as many as we have
Beth
Time for okay so first question for you from the audience how does certify and know if an error occurs and how does it recover
Avinash Tiwari
After that yeah so it has the I can't reveal the algorithm in detail but it has the capability to analyze certain errors and then
Avinash Tiwari
It starts the flow flow from where the error was done so it ends the flow there and then it starts from the next flow
Avinash Tiwari
That it has detected so it first detects all the flow which are possible in the application and then goes through each of the flows
Avinash Tiwari
Sequentially and if there is an error within the flow that flow is stopped and the next flow is executed and if the error is
Avinash Tiwari
Because of certain crashes it captures the crash which is then reported back to the user in form of the crash test but if the
Avinash Tiwari
Error is more functional in nature you will see that in the snapshots where you will have to analyze why that error occurred in future
Avinash Tiwari
We are trying to figure out can certify itself point out why'd that ever happen but right now the user based on the screenshots and
Avinash Tiwari
The workflow which was executed will have to analyze it that's a functional error or because of a crash and things like that
Beth
Okay thanks for that extra information about certifier this person wants to know can you inject dynamic parameter values from other test streams
Avinash Tiwari
So right now no so as you saw either we can provide the test data while we are configuring the test execution so however it
Avinash Tiwari
Can be configured in form of certain files or a JSON object but if by another test stream means through some other tools not right
Avinash Tiwari
Now it has to be within the certifier system and within the configuration files that we provide right now but that's a good point if
Avinash Tiwari
It can be linked to some may be external dbe's or external systems
Beth
Okay good to know thanks this person says continuous testing is all about automation do you have any data points on the industry average of
Avinash Tiwari
The percentage of testing that can be automated good questions so
Avinash Tiwari
Of what we have seen from our experience and it varies from application to application what we have seen from our experience is a lot
Avinash Tiwari
Of mature organizations have been able to automate almost 60 to 70% and those organizations are following a practice called automation pyramid so the reliance
Avinash Tiwari
On functional automation tests UI driven functional automation tests are less because those are more brittle in nature and specially on mobile apps the chances
Avinash Tiwari
Of these kind of tests getting failed on different device is very high so they follow approach where they maximize the unit level testing then
Avinash Tiwari
API level testing and then UI driven functional test so combining all three they have been able to achieve almost 60 to 70% of the
Avinash Tiwari
The test coverage and as the tools are maturing more to mimic the real life situations the percentages will increase for sure and again use
Avinash Tiwari
Of PID systems will further increase it so I'd give a try it now I can say 60 70 % is what we see as
Beth
Industry approach ok great thanks this is a popular question I think among testers this person says why would you think that a I wouldn't
Avinash Tiwari
Replace QA resources so yeah this has been interesting debate lot of different viewpoints but what we feel is the end of the day AI
Avinash Tiwari
Also has to be trained so for example we saw we still need to provide test data to the system and when the reports are
Avinash Tiwari
Generated though AI a is going to analyze and provide report on its own but someone has to be analyzed whether that's a third issue
Avinash Tiwari
Localize the bug and then report it so I think the need of QA is still going to be there however I think the QA
Avinash Tiwari
Teams have to really scale themselves with other as you saw we generate so much of data and until unless you really understand what the
Avinash Tiwari
Data means so you have to really go deep into how application works for example if there is a CPU spike why that is happening
Avinash Tiwari
So so those kind of skill improvements are definitely required but I mean the QA has to be there I mean a lot of organizations
Avinash Tiwari
Are merging Devon QA together that's definitely happening but the testing will exist but it will become more tool based where the teams will drive
Avinash Tiwari
The tools analyze the results and then take smart decisions
Beth
Okay great thank you and I think that's a great place to end to so that will end our web seminar for today I'd like
Beth
To thank a vanished owari for his time and pee cloudy for sponsoring this webinar and of course a special thank you goes out to
Beth
All of you in the audience who spent this hour with us today so everybody have a great day and we hope to see you
Beth
At a future event thank you thanks everyone for attending this seminar
