Dinakar
Well, hello everyone and welcome to Pcloudy's webinar on root cause analysis for mobile app performance issues.
Dinakar
My name is Dinakar and I'll be the host for today.
Dinakar
Before we get started, I would like to go over a few housekeeping rules.
Dinakar
You may have noticed that your line is currently muted.
Dinakar
If you do have any questions during the webinar, you can use the Q&A option and submit your questions during the webinar there and we'll get to those answers.
Dinakar
is we'll get to the answers at the end of the webinar or if we're able to we'll do it while we're doing the presentation itself.
Dinakar
Uh please note that this webinar is being recorded and we will send out the link of the recording to share with your colleagues later.
Dinakar
With that said, let me introduce the speaker for today, Jessica Selvin.
Dinakar
Jessica is a pre-sales specialist at Pcloudy.
Dinakar
She has around four years of experience in the IT industry.
Dinakar
uh prior to her role at Pcloudy, she has worked with uh giants like LTI Minry as a pre-sales consultant in automation solution design and for various other industries as well.
Dinakar
She has a keen expertise in data analytics and various automation tools.
Dinakar
So without further ado, let me hand it to our speaker Jessica Selvin.
Dinakar
Over to you.
Jessica
Thank you.
Jessica
Thank you so much Dicker for that uh introduction.
Jessica
Um so I hope my screen is visible and I'm audible.
Dinakar
Absolutely.
Dinakar
Yeah.
Dinakar
Great.
Jessica
Uh so as Dakar mentioned our topic for today is having doing a root cause analysis for the various performance issues that you would encounter while testing your mobile application.
Jessica
uh right and how would we go about doing that and what are the road obstacles that are typically faced by QA teams uh when going about handling performance issues in mobile applications right so that would be the focus of our discussion today and we will be wrapping up with a live demo as well uh so uh firstly and to uh what you might relate to even on a day-to-day basis as you use mobile applications right uh You will be able to see that uh some here are some stats before you.
Jessica
All right.
Jessica
So studies show that 65% of users uh lose trust after just two app crashes.
Jessica
So this is a this is huge because trust is very hard to regain and a stable app uh in today's world is not just a technical goal.
Jessica
It is a business priority.
Jessica
Uh right.
Jessica
So and even as a personal uh user of many applications you might have faced uh when you encounter issues like this it does cause you to uh lose trust and uh abandon the application.
Jessica
All right.
Jessica
Uh next is the page load time.
Jessica
Uh the other stat is that for every second of page load delay uh can reduce conversion rates by roughly around 7%.
Jessica
And this is particularly uh you know impactful in e-commerce banking sectors which might lead to direct revenue loss.
Jessica
Right?
Jessica
So a fast loading app encourages higher user conversion rates uh for your application.
Jessica
Um and thirdly you can also see seven out of 10 users prefer app sizes less than 50 MB.
Jessica
uh right so uh especially in markets where data and storage are constraints keeping the app size in check ensures that you have a smoother first-time experience uh and does not uh you know strain the device so this is another key factor so I'm just listing out some key factors as a user you might you might be able to relate to this as well right and uh the app launch time so there is a higher bounce rate and app abandonment rate when the apps takes takes more than 5 seconds to launch.
Jessica
Uh right.
Jessica
So this would lead to the user retention coming down.
Jessica
And finally on the scroll performance as well.
Jessica
Um it is critical for user engagement uh and for users to be retained on your application.
Jessica
So even uh incremental improvement in scroll performance can yield a 3% increase in user engagement.
Jessica
Right?
Jessica
So small optimizations can lead to significant u benefits over time uh can and which will directly lead to uh a good business impact right so this is uh so no longer as I mentioned no longer is a stable app something just to a technical achievement it is a business priority all right so with that context uh let's just see a further deep dive into what are the customer expectations of today all right with the complex digital infrastructure that we have uh your app will have uh you know multiple layers.
Jessica
So uh be it the UI layer in which your user base is accessing the application uh be it from mobile phones, tablets uh right and the middle layer where uh the app uh thinks and talks to other services right so for example managing uh security and authentication firewall and integrating with other services and finally at the bottom and the backend layer we have the databases the internal and external APIs uh and the core systems of your and architecture of your application right so with this complex digital infrastructure of your application.
Jessica
Uh the customer expectation still remains high, right?
Jessica
With um as here are some popular examples, right?
Jessica
They expect it to be as seamless as Instagram and as fast as Google, smooth and secure like Amazon and Apple, right?
Jessica
So this is the type of expectation uh users have for uh their any application that they're using and they're comparing it to these applications and uh they're looking for a flawless functional experience uh right a lagfree performance experience uh right so uh as well as 247 availability.
Jessica
So uh ensuring that every feature works reliably each time and uh as we had already mentioned right sec even seconds of delay can cost in your conversion rates and uh finally a perfect uh visual experience where all your animations are rendering smoothly the page transitions are smooth and so on.
Jessica
So these aren't again uh features anymore they are expectations and becomes a part of your user base's expectation.
Jessica
All right and your app uh becomes your brand.
Jessica
So uh the users would be equating the quality of your app experience with the quality of your brand itself.
Jessica
Right?
Jessica
So that is the kind of expectation uh that's there.
Jessica
So with this expectation uh and uh user experience that we see of user experience what are the why do app performance issues keep coming up right this is something uh and especially coming up in the later stages postp production even uh right so firstly is the lack of identifying real bugs so symptomatic treatments or only seeing surface level issues and unable to identify what are the real uh bugs right?
Jessica
Not enough testing coverage.
Jessica
So limited test coverage uh be it in different scenarios uh or be it testing on different uh devices, OS versions uh different uh location, network conditions, right?
Jessica
Being able to provide real world context and being able to test in different scenarios.
Jessica
So this is also something that is uh challenging.
Jessica
Uh next is access to um meaningful data right so there is a lot of fragmentation multiple sources of information uh right which is fragmented uh maybe spread across multiple uh teams even uh right so this makes it difficult and the data that the type of data that you get uh it's also difficult to infer and get insights from that data and it's hard to get those precise metrics like be it your response times your CPU memory that's also aligned with actual user behavior.
Jessica
Right?
Jessica
So this is a core issue.
Jessica
Not having access to meaningful data, not having a single source of truth uh for your information and not uh it being fragmented across different teams making it difficult to collaborate.
Jessica
So this this forms uh the core one of the core issues as to why this keeps coming up.
Jessica
And finally uh a single performance bug can cause cascading uh failures right like the domino effect making uh root cause identification even more harder.
Jessica
So QA teams would be finding it difficult uh to trace errors precisely across the app's architecture and struggle to pinpoint exactly where um it started and how it started impacting other areas.
Jessica
So this is another issue that is faced.
Jessica
Now let's just take a step back and see what is the ideal way to do defect management right and how is it supposed to be done uh with this context.
Jessica
So firstly it is finding and documenting the bugs uh right uh defect identification uh grouping and categorizing your defects and as well as prioritizing making it easy for teams to understand uh the uh you know uh what to fix first and then developers get into fixing the issues based on the priority and ensuring that they need to get it to the root cause and not just the symptom ensuring that this issue is not a recurring issue.
Jessica
uh then QA validates and verifies this uh testing to confirm that it resolves the issue and it doesn't introduce other new bugs right and the issue is closed and it's retested across different uh scenarios and uh ensuring that the coverage is high and ensuring that this app is stable across different uh environments and devices.
Jessica
So this is the ideal way in which defect management uh should be done.
Jessica
But what are the kind of realities that we see uh with defect analysis right uh firstly we see a band-aid approach uh right so uh be it when though the uh process looks seamless the reality is very different and there's a lot of challenges right uh defect fixes end up being quick patches rather than long-term solutions and teams are focusing on making the error disappear or the symptom disappear without uh addressing the root cause and um yeah like I said fixing the symptoms uh the deeper problem often goes uh undetected leading to bugs resurfacing later on uh causing further delays uh and again key to the issue is not enough data like I mentioned not having actionable insights not having um data that would actually reveals the why behind the issue right without meaningful metrics you can only guess where things are going wrong without actually being able to do a proper RCA.
Jessica
Uh inability to replicate errors.
Jessica
Uh right.
Jessica
So um this makes it harder to validate and to fix this and teams uh continuously being stuck more in the reactive mode issues being found in the production.
Jessica
Uh right and only when a certain symptom is found then um reacting to that and uh doing a symptomatic solution to those uh issues.
Jessica
Right?
Jessica
So this is the reality that we see uh in defect analysis and u with that what are the cost of defects that we see and especially in what stage of your software development life cycle uh do you see these issues coming up and uh what is the cost in the different stages right so the earlier you detect and fix your bugs the cheaper it is going to be uh right so for example at the requirement and design stage it is the least impactful ful in terms of your cost in the cost impact uh and it would be nearly negligible and the cost would increase as you move forward uh being the highest in the production right once it's your your application is deployed the cost both in terms of uh your brand reputation as well as financially the identifying performance-based issues post-p production would be really high cost to you right so it is imperative to prioritize is uh doing performance not just functional validations but performance validations for your application before uh it is uh you have deployed your application right and not just in the context of load testing but at the app level at the user level uh to see how is it actually um for an individual user to use your application and what are the back-end metrics that might be uh causing the user experience to be impacted.
Jessica
So that as well is critical uh right.
Jessica
Um so the focus should become more and more shifting uh the testing left or shift left testing making sure that we detect the performance issues address the performance issues as early as possible uh when it's easiest and as well as cheapest to fix.
Jessica
So yeah, this is uh a humorous way of actually illustrating what is the problem when you symptomatically treat something.
Jessica
Uh right.
Jessica
So um be it in at in any personal life situation as well as what we see in QA, right?
Jessica
Um with root cause analysis, um ensuring that you're actually focusing on the cause and not just the symptom or the effect.
Jessica
Uh right.
Jessica
So um and this is a and having a structured way to find out why a problem happened and uh RCA uh is a mind mindset shift right and the goal should be to prevent the recurrence.
Jessica
So ensuring that you start with the symptom.
Jessica
Yes, you start with the apparent problem but you get to the deeper uh root causes uh have the possible root causes and then you finalize on the actual root cause and deliver a solution that ensures that it doesn't uh repeat again in the upcoming versions of your build.
Jessica
Right?
Jessica
So that should be the goal uh and the mindset shift when it comes to uh root cause A few other challenges that uh you know come along with mobile app performance and what makes it tricky is firstly device fragmentation.
Jessica
All right, you need to ensure that you test across uh different devices.
Jessica
You might have seen your app work on one particular device in one version.
Jessica
Uh what is the uh ensuring that it's covered across different versions and uh different builds and different uh scenarios, right?
Jessica
So be it your device type, screen sizes, OS versions, um hardware specifications.
Jessica
Uh so it becomes challenging to test reliably across the entire mobile ecosystem.
Jessica
Uh next varying network bandwidth or for example in different network uh conditions.
Jessica
Um so the app may work well in a strong network but may degrade sharply uh right when the connectivity is u lagging or limited.
Jessica
Uh next complex user journeys right so modern mobile applications have highly dynamic user flows uh right with different steps and longer uh critical user journeys right so testing uh every path to spot those bottlenecks is also critical but uh challenging and finally tool fragmentation so to get the data tool Q QA teams are using fragmented tools uh for different test scenarios uh be it for performance ments and another for functional testing and another for analytics.
Jessica
So this will lead to delay inconsistencies in result um different teams not collaborating uh right and um not having enough data to make a correlated uh decision uh and will affect the data interpretation.
Jessica
So these are some of the things that will make u mobile app performance uh tricky in particular right so how can we resolve this and how has Pcloudy strategized this uh solution right so that's where we're getting into the solution part um and Pcloudy has been forefront of these kind of trends and uh innovating particularly in the AI uh app testing space right so uh to accelerate growth and to be innovative uh in terms of the strategy particularly in the mobile app performance space.
Jessica
So firstly the solution is to measure the right app performance indicators.
Jessica
All right ensuring that uh you have your uh CPU data your memory and all these different metrics.
Jessica
So I have just listed out a few here but there are um 60 plus metrics that Pcloudy's app performance dashboard will give you.
Jessica
All right, ensuring that it is presented in such a way that is sharable that can be easily correlated uh right and it's easily understandable right and it does not require multiple sources of uh truth right so that is something that is um really important in ensuring that you have the right data and a right uh framework within which you're looking at uh implementing a solid test reporting strategy Ensuring uh having a quick overview of the data, having easy to understand reports from a single source.
Jessica
Uh having detailed logs and ensuring that the reports are easily sharable also having access to the UI side like be the corresponding screenshots and also being able to easily correlate the screenshot with the metrics um access to the video and integration with other bug tracking tools.
Jessica
So these are just some of the uh key points uh that you need to have in mind while having a solid test reporting strategy particularly on the performance side.
Jessica
Next uh how uh Pcloudy is flipping the script with performance experience testing right so be it yeah sorry so we are tracking as I mentioned 60 plus metrics and we're monitoring everything from your CPU your battery to your UI metrics to your network uh data right across platforms having realtime data collection so we have a real device data infrastructure so that makes it much easier to get the real time data and capture ing as they happen without any synthetic delays or sampling errors.
Jessica
Right?
Jessica
So ensuring that you have real time uh data collection and advanced data processing.
Jessica
So we have AI working behind the scenes in order to categorize the data to give you a IML based insights based on the metrics collected.
Jessica
So let's just see um on a high level the different layers of the solution.
Jessica
The first and foremost at the bottom you see is our diverse real device infrastructure.
Jessica
So we have uh 5,000 plus device browser combinations and particularly on the mobile side we have hundreds of real devices that are hosted in our data centers.
Jessica
So these are real devices not simulators or emulators.
Jessica
These are physical devices hosted in our data center.
Jessica
So this ensures that you can create and run your performance test as authentically uh right uh in a real world testing scenario and testing across.
Jessica
So we support all the latest versions uh right uh both on Android and iOS for you to test on across different OEMs like Apple, Google, Samsung uh right and several others.
Jessica
So uh you will have uh real true diverse coverage uh for you to do your performance runs and secondly uh you have the performance data collection engine that will u you know go through your the flow of your application.
Jessica
So at the user side you will simply be doing a critical user journey for example.
Jessica
Uh but at the back end there will be uh all these different data points being tracked and it will be real-time data collection right and thirdly you have the visualization engine uh where you will be uh it will take all these metrics and plot them into charts and graphs uh that are easily understandable to the user.
Jessica
And finally we have ML based categorization uh anomaly detection uh right that will give uh analyze those metrics and give you the analysis ML based analysis of the same right so this is these are the layers of how our app performance testing module works and with that we will get into the live uh demo just give me a second to share my screen on Yep.
Jessica
So this is Pcloudy our platform uh right where uh as I mentioned we are a digital experience testing platform and our focus for today is the app performance module but uh other than that we support uh functional testing automation testing integrating your frameworks with our devices uh both on our devices um uh mobile devices as well as browsers we also have an AI agent Qpilot um but uh our focus for today is our app performance performance.
Jessica
So let's just get straight to the app performance […]
Jessica
So this is the device list that we have, right?
Jessica
These are the real devices that I was mentioning that are hosted across in our data centers.
Jessica
Uh you can connect to anyone uh and uh begin your app performance testing.
Jessica
So prior to that uh we first need to ensure that our application our APA or APK or IPA file is uploaded into the my data section.
Jessica
Uh so this is where you will be uploading your bills to do your testing.
Jessica
Right.
Jessica
So you will simply need to click on upload file and upload your application into uh the my data section here.
Jessica
And for performance testing there is one additional step that needs to be done.
Jessica
Uh where for example you select the application that you want to uh do your performance run.
Jessica
And under actions you click on app performance testing.
Jessica
And what this will do is it will instrument your application to make it ready for it to uh record and collect all those different data metrics.
Jessica
Right?
Jessica
So uh here you can see that I have already instrumented the application.
Jessica
It has added the relevant libraries in order to collect the data and uh it has renamed the application as underscoreerformance instrument and you can see uh you will be using this application in order to test right.
Jessica
So with that let's just get back to our app performance module.
Jessica
So once we've done that we're ready to begin our testing.
Jessica
So I'm going to go ahead and select a device of my choice and I'm it's fetching all the applications from the data section that we have just now uploaded.
Jessica
So I'm just selecting this particular application uh to do my testing and I'm going to click on […]
Jessica
So firstly the device session is getting established and uh with Pcloudy uh as soon as you click on connect you will have complete access of the device with you uh and there is no session limit as such you will be able to go ahead and uh do your testing unlimited all right so here it's asking for u creating a tag that is in case if you want to compare sessions for now we will skip that um and also you can see here that the launch times are getting measured so before we begin our recording of the metrics.
Jessica
Uh the app is going to get launched three times uh to ensure that it's measuring the cold, warm and hot launch time.
Jessica
So you will be able to see this in the report and um we can see the metrics measure the metrics there.
Jessica
So once it has launched three times, so it'll take around 30 to 50 seconds.
Jessica
Once it has launched those three times, the recording button is now enabled as you can see and I can begin my recording.
Jessica
So this is just a sample application where I'm demoing on.
Jessica
Uh right and we can begin the recording.
Jessica
So right now um I just don't need to bother about the data at this point.
Jessica
I am just going to go ahead and do any critical user journey.
Jessica
Right?
Jessica
So uh you don't need to worry about what's going on behind the scenes.
Jessica
Just simulate um or mimic user behavior.
Jessica
So that is the focus at this stage during your test.
Jessica
So I'm just uh going to do a very small simple flow […]
Jessica
So right now uh while I'm just mimicking the user journey behind the scenes um uh the data collection engine that I was talking about right that layer is working and collecting all the data behind the scenes.
Jessica
So I'm just going to add an item to the […]
Jessica
We'll maybe add another item to the cart and be done with our flow.
Jessica
So yeah um I'm just doing a short flow in the interest of time but there is no uh session limit as such you can go about doing your entire critical uh user journey along flow whatever is required and go ahead and release the device.
Jessica
So I have now released the device.
Jessica
So we can now get into the report section here where we'll be able to see the reports that have been generated.
Jessica
So you can see here this is the report that's generated.
Jessica
Let's get to it.
Jessica
Yeah.
Jessica
So this is the app performance report that has now been generated.
Jessica
Right.
Jessica
So firstly we are in the summary page uh where you can see that the app deck score has been generated for this.
Jessica
So you can see that uh it has given several different categories from poor, fair, good to excellent and this is a value between zero to one uh right and uh this is basically a metric to measure the response times of your application.
Jessica
So if the response time is smooth and seamless it will be towards one.
Jessica
So you can see that this is a pretty high good score and it's coming under the excellent category.
Jessica
uh this is measured mainly between the response times of your APIs between how many are within the threshold and how many are slow and based on averaging them out it has come up with this um appex score uh below you can also see the AI based uh insights that are generated so it has detected a slow web page load at one point uh right it has also identified janky frames so these are frames uh that fail to render smoothly right and it crosses the thresh threshold allocated to it.
Jessica
So that's typically if it crosses more than 16 uh milliseconds, right?
Jessica
So here it has captured 785 frames that are janky.
Jessica
Um you it has also identified that there are some failed API requests and unoptimized resources.
Jessica
So it has summarized these particular issues two from the UI category and uh a couple from the network uh category.
Jessica
And above you can see all the device related information.
Jessica
So you can see uh that uh the device name the specifications of the device uh right as well as the session related information with the timestamp and the duration of the session.
Jessica
Uh you can also see the application related details like the package name, the version, what is the build number and the app size.
Jessica
You can also see the app related insights right.
Jessica
So for example the installed app size uh what is the maximum C or the peak CPU usage uh right that is consumed during the session uh averaging out what is the memory usage of uh the app during the session and the average CPU usage and this is what I was referring to uh with the different launch times right so a cold launch uh occurs when your app is started for the first time after you reboot your device right and it's not in the memory so the first launch it has happen pretty fast in uh less than a second.
Jessica
And the warm launch is when it is still in memory but uh the activity has been destroyed.
Jessica
And hot launch is when uh it is typically the fastest type when it's occurring uh it's simply brought from the um background to the foreground.
Jessica
So that is on the different uh app launch times.
Jessica
You can also see the UI related insights over here uh like uh the jank frames uh the average frames per second uh which will give an idea of the smoothness of your application as well as the stability of your application.
Jessica
You can also see the total number of API requests and what is the error rate of your APIs and what is the average response time of your APIs along with the peak response time.
Jessica
Right?
Jessica
So the peak almost took 3 seconds to respond.
Jessica
So these are the kind of insights uh highle overview that you will first get by scanning our summary uh section of our app performance report.
Jessica
With that we can go ahead and deep dive into the other sections as well.
Jessica
So firstly let's get into the app insight section.
Jessica
So let's get there.
Jessica
Yeah.
Jessica
So this is our app insights which will give you a time series metrics of 20 plus metrics uh represented graphically across the duration of your session.
Jessica
So you can see you can scroll through the entire duration of the session along with giving you what is happening correspondingly in the UI right and I can uh scroll this u you know time series to get to the particular time stamp that I want to analyze and I'll be able to uh correlate the different metrics here.
Jessica
Right?
Jessica
So you can see here that for example at 1 yeah at this particular time stamp uh their application CPU is at around 4%.
Jessica
But if I drag it out maybe a little more here this side I can see that uh there is a one janky frame that's detected uh right uh along with um high memory consumption of 66 MB in the Java heap and 184 MB in the native heap.
Jessica
Right.
Jessica
So I can scroll around and see that this was during the search for a particular item to add to the cart.
Jessica
Right?
Jessica
So you'll also be able to correlate the UI and all the different metrics.
Jessica
You can see that uh we provide the total frames along with how many frames were janky.
Jessica
So at this point there were 71 frames.
Jessica
You can see and correspondingly there were uh six frames that were janky and the others were smooth.
Jessica
You can see the smoothness percentage along with what is the count of frames uh that missed the deadline.
Jessica
You can also see uh the GPU, the total memory usage along with the Java native code heap uh right and you can also see the latency count uh what is the count of the slow UI thread as well as you know you'll be able to see the transmitted uh on the network side right so you'll be able to see the transmitted packets the transmitted bytes uh received data as well and finally on the battery drain voltage and temperature so all these different metrics 20 plus metrics are presented graphically uh for you to analyze uh along with the corresponding UI.
Jessica
So this is a key differentiator with Pcloudy right um no other uh cloud service provider uh or cloud infra testing platform is giving this level of detail into um you know the performance metrics and giving you this level of analysis details right so typically you would have to to get this type of data you would have to go to different data source sources and have multiple uh tools in order to analyze it.
Jessica
So here we are giving it to you in a single uh page and from our end we have simply just mimicked a user journey right so there is no other technical expertise as such uh required in order to use the tool next is our UI insights so here you'll be able to see the page transition time between every single page how much time it took to transition and again you'll be able to see the corresponding uh UI what is happening right and if there is a particular slow load it is able to detect that and it is marking it as a slow load as well, right?
Jessica
And you'll be able to see it uh along with the corresponding video, right?
Jessica
So here you'll be able to uh detect each and every page transition.
Jessica
And finally, uh sorry, yeah, this is the jank analysis where you'll be able to see the total of number of frames and all the frames represented in a timeline.
Jessica
So you can see how many were on time, how many uh were uh delayed and how many might have caused uh uh went beyond a certain threshold might which might lead to a a lag experience to the user right and if there is any dropped frame that will also be denoted.
Jessica
So you can hover over any particular janky frame and you'll be able to see what is the type of jank uh right and at which layer.
Jessica
So this was at the product main activity.
Jessica
So this is when the product search was going on.
Jessica
All right.
Jessica
So you'll be able to share all these details with your relevant stakeholder teams right and as I mentioned these reports are completely sharable.
Jessica
So this link is sharable uh across to different team members and they can also go ahead and access these reports.
Jessica
Uh you can also see what is the maximum type of errors uh in the jank analysis along with the count and the percentage of the errors as well.
Jessica
And finally on the app activity visualization uh you'll be able to see how many total screens uh were changed right along with the activity name.
Jessica
So you can see there's a uh pop-up activity there's a home activity um the cart search searching for the products and all these different activities right along with the performance.
Jessica
So you can see based on the legend here that um all these were fast and this was uh in the normal range along with the performance insights right.
Jessica
So you can see here uh what was the activity that had the highest load time uh what were the activities that were repeated and what is the overall flow of the application uh where it's going through multiple screens uh eight screens and returning to the main activity.
Jessica
So this is the uh activity visualization page and uh next uh very importantly is our network data right.
Jessica
So uh this is another uh key feature and a key differentiator where we are providing the details of all the API calls uh that were made during your session uh right along with the analysis.
Jessica
So firstly you'll be able to see the status and the count of your uh the APIs with different statuses.
Jessica
Uh right so you'll be able to see the status distribution and breakdown along as well as the timing uh distribution and breakdown.
Jessica
Right?
Jessica
So this is something you'll be able to see and then you'll be able to see each and every API call that was made and on the left hand side are the categorization uh issues right so it has categorized um it has analyzed each and every API and categorized it into different issues so you'll be able to see what are the failed requests right so you can see there's a failed request over here uh you'll you can go ahead and click on the slowest endpoint uh and you'll be able to click on each each and every uh API.
Jessica
So for example, you can see this nearly took 3 seconds.
Jessica
Uh you can also click on every individual API and see the response body and the timing breakdown and at which stage uh was most of the time consumed.
Jessica
All right.
Jessica
Similarly, you can see if uh any images were loading in an unoptimized uh format which might lead to any performance issues.
Jessica
You can also see APIs that are missing cache headers.
Jessica
Right?
Jessica
So cache headers is basically it'll ensure that that particular data is cached and it doesn't have to send uh the API request again.
Jessica
All right.
Jessica
So and similarly if there are no cache directives as well for the same reasons to ensure that the API requests are not getting repeated unnecessarily.
Jessica
So you'll also be able to see what are the cache uh directives that were missing.
Jessica
You can also see if there's any images and static images that are not uh cached as well.
Jessica
And finally you'll be able to see what were the repeated API calls.
Jessica
So here you can see that this particular API call uh was called seven times uh right along with the count and the details time stamp of those API.
Jessica
So this particular API was called uh 36 times.
Jessica
So this is a relevant data where uh can be shared with the developers and the relevant teams to see whether the these API calls can be optimized.
Jessica
Right?
Jessica
So this these kind of things have a direct performance imp u impact on the user.
Jessica
Right?
Jessica
So looking for opportunities to optimize, right?
Jessica
And having the root cause uh presented to you in directly in this format really helps to uh make your uh defect management process much more faster.
Jessica
So this is on the repeated API calls.
Jessica
And finally, you'll also be able to see the domain sharding or the domain distribution.
Jessica
So you can see here that the total requests uh is 823 uh with 59 unique domains.
Jessica
And you'll also be able to see what are the distributions of these domains.
Jessica
Uh right.
Jessica
So you can see that uh primarily and majorly 66% of the API requests were pre uh were sent to this particular domain and you'll be able to do that analysis as well.
Jessica
So this is u on the network analysis, right?
Jessica
So you'll be able to uh drill down into each and every individual API and also see the header information.
Jessica
So that is also something uh that you can uh share with the development teams right so you'll be able to see the header information um preview the response and the timing breakdown of your API and finally we have the log analysis so firstly it'll provide you the device logs for you to download and view uh and secondly we have our log analyzer engine that will categorize uh go through the entire logs and categorize it into different issues.
Jessica
So you'll be able to see you can for example here it has identified an issue see that relevant log lines along with it.
Jessica
So you don't need to uh go through uh you know line um many lines several lines of log uh um and analyze it right so here the AI categorizes it into different issues.
Jessica
So even if there's any crash, if there's any ANR or app not responding issues, you will be able to see those relevant log lines as well that are categorized here.
Jessica
So this is um just to give you a brief on our uh app performance.
Jessica
This is the kind of data that you'll be able to see.
Jessica
This is the kind of correlations you'll be able to do with our uh app performance uh engine.
Jessica
Uh right.
Jessica
And finally there is one more thing that I wanted to touch upon which is our compare feature.
Jessica
So you can also compare between two reports.
Jessica
So for example here uh say you have done two different sessions.
Jessica
So in this case um the it was on the same device but two different versions of the build are compared to see if there's any performance difference between the application.
Jessica
So you can see here that the launch times are double from session A.
Jessica
So the second version uh took a little longer to load.
Jessica
So the first one was 1 second and the second one took around two and a half seconds.
Jessica
And similarly here as well it has doubled.
Jessica
You can also see that it has around 13% more uh issues that have been identified uh like the response time is slightly uh higher and as well as the CPU consumption is uh more higher in session B.
Jessica
So these are the kind of data that you'll be able to compare between two sessions as well, right?
Jessica
So for example, if you want to compare between um different specification devices, right?
Jessica
One on a higher end and one on a lower-end device, you'll be able to do that and see if there is any performance impact uh for your application.
Jessica
So for example, you can see uh the battery drain is slightly higher in session B compared to session A.
Jessica
Uh right.
Jessica
And similarly on the janky frames, there are a high number of janky frames detected in session B over session A.
Jessica
So this is the type of analysis.
Jessica
You'll also be able to do comparative analysis between two sessions as well.
Jessica
And you can also see the application uh information.
Jessica
So you can see that uh the app size is much higher uh between the two devices which are two applications which could have contributed to some of the uh performance issues along with you can see that the devices were same.
Jessica
So you can see that the comparison was done between two different builds of the application.
Jessica
You can also have the video timeline.
Jessica
So you'll be able to see the video of the entire two sessions and compare them side by side.
Jessica
So that is on our comparative analysis.
Jessica
So hope this gives you a a good and a broad idea on the kind of analysis you will be able to do and achieve.
Jessica
What are the kind of root causes you'll be able to easily uh uncover with our app performance engine.
Jessica
and hope the session was insightful.
Jessica
So with that um we will h hand it over to the Q&A session and let's take it from […]
Jessica
So um yeah uh are there any questions?
Jessica
Dakar, are you able to see any?
Dinakar
Just uh just a few ones.
Dinakar
I can see some of […]
Dinakar
Are you able to see them or shall I read them out or just […]
Jessica
Yeah, it would be helpful if you could maybe read them out or Yeah.
Dinakar
Yeah.
Dinakar
I think I can see them now.
Jessica
Okay, great.
Dinakar
So, how can I identify and quantify performance regressions between bills?
Jessica
Right.
Jessica
So yeah, this is where your two things would come into place.
Jessica
Another thing that I missed to mention is that we support automation as well, right?
Jessica
So I had manually done a flow, but say for example, you want to run a performance test from your application, sorry, your automation scripts, right?
Jessica
So you can, for example, if you have an APM script, you can go ahead and uh run it uh and enable a particular capability called app performance true.
Jessica
All right, we can share you the documentation on that uh if you're interested and you can simply run your automation scripts on Pcloudy devices and you'll be able to do the comparative analysis.
Jessica
So there is no need to manually access in order to do a performance run.
Jessica
So that is in that way you'll be able to uh do it between easily compare between builds.
Jessica
All right.
Dinakar
Is there a benchmark for app performance?
Jessica
Yes.
Jessica
And we have uh documentation on this as well.
Jessica
So uh you you can see that for for example for a janky frame it's whether it's beyond if it crosses 16 milliseconds that is the benchmark all right and for that there are several uh you know industry standards and thresholds uh we have detailed documentation on that we will be happy to share that with you.
Dinakar
How can we manage variations in device models, OS versions, and network types?
Jessica
So yes, so uh as I mentioned, you can choose uh the devices of your choice uh right to compare between different um devices in order to see how the performance is changing.
Jessica
And because Pcloudy has um real device infrastructure, this will really help in test coverage and ensuring that your test coverage uh is across different uh you know devices and see whether there's any performance impact for those […]
Dinakar
Next, um, does it support for iOS?
Jessica
Yes, we I have shown this uh for an Android application for this webinar, but we support the same and it would be pretty much the same way for iOS as well.
Jessica
Um, there are just a few metrics because of the limitation on iOS side uh that you won't be able to uh test it, but yes, typically you will be able to do the exact same performance run for iOS based applications as well.
Dinakar
Can we test the app in different network bandwidths like 3G, 4G and 5G?
Jessica
Yes, we are uh providing a network simulation where you'll be able to uh simulate this.
Jessica
Yes.
Dinakar
So I think uh I'm not able to see any further questions.
Dinakar
I think um most of the questions were covered.
Dinakar
If there are any that I've missed then please let me […]
Dinakar
This is the quick one I think that's come in.
Dinakar
Uh yeah.
Dinakar
Okay.
Dinakar
Okay.
Dinakar
As a user, how much time I need to use like 20 or 30 minutes application to collect and give realistic data.
Jessica
So um more than time uh I would say it is based on the journey right like a to perform a critical user journey like for example right from onboarding a user for example or adding several items to the cart and removing that and getting to the payment page.
Jessica
uh right like these kind of it's more so of the ensuring that you do a complete workflow that will give you a good uh idea right so there is no session limit like be it whether it's a 5 minute session or 30-inut session um the metrics Pcloudy will give you uh all the uh relevant metrics but it's more important that you do a realistic uh customer journey to give you that uh data so it's not so much time dependent as much as you do the uh required flows to get the right […]
Jessica
Hope that answered the question.
Dinakar
I think maybe one of the questions that that's will be helpful for the audience is could be, you know, does this tool kind of work with other CI/CD tools or like defect management?
Dinakar
Can I integrate this with my defect management tools where I can see everything in one place?
Jessica
yeah yeah yeah so if you want to report any bug right so we already do have uh integrations with CI/CD as well as bug tracking tools like Jira Trell Slack and so on uh not yet in the performance module it is not yet live but this is definitely something we can easily bring about uh for performance as […]
Dinakar
All right, I think that's mostly it.
Dinakar
Uh, I don't see any more questions coming […]
Dinakar
Yeah.
Dinakar
Yeah.
Dinakar
Yeah.
Dinakar
Yeah, I think we've answered most of the questions, Jessica.
Dinakar
I think we're good here.
Dinakar
All right.
Dinakar
Thank you so much, folks.
Dinakar
Uh before I end the webinar, I would like to thank Jessica for you know sharing your wisdom and this whole field of app performance experience testing and you you know the way you've done a deep dive into the root cause analysis and showing us how this performance testing can be done and assist us with our you know root cause analysis.
Dinakar
So thank you uh and also we want to thank each one of our participants for joining us for this webinar.
Dinakar
We're sure some of the ideas some of the insights that were shared by Jessica may have sparked a few few ideas to think through.
Dinakar
Uh my recommendation would be to mull over these ideas, talk to your teams, talk to your you know managers, reach out to us if you want to discuss something further and you know keep the conversation going.
Dinakar
Also feel free to reach out to us at any time.
Dinakar
Uh you can write to us on email uh or you can go out to our website and use the chat option u and somebody will be right with you and or you could even message us directly on LinkedIn.
Dinakar
Uh our handles are available on our websites as well.
Dinakar
So you can reach out to us on LinkedIn, reach out to Jessica on LinkedIn as well.
Dinakar
And yeah, so thank you so much.
Dinakar
Thank you for being a patient audience and also an engaging one with all those questions and happy […]
Dinakar
Thank you.
