August 4th, 2016 by Avinash Tiwari
Gartner’s Market Guide
Delhi, June 23, 2016 – Smart Software Testing Solutions (SSTS), a leading provider of Mobile App Test Automation Tools, today announced that it has been identified as one of the Representative Mobile App Functional Test Automation vendors in the Gartner’s Market Guide for Mobile App Test Automation Tools(1).SSTS was named in the category of commercial vendors with functional test automation and device cloud capabilities.
According to Gartner, ‘’the need for automation in mobile app testing is driven by business time-to- market demands and digitalization initiatives. These push mobile app development teams to adopt agile development approaches to deliver high-quality mobile apps faster. This rapid pace, combined with a broad and changing device ecosystem, creates a challenging environment in which to maintain a high level of quality. Without automation, it will end up crushing all but the most trivial app efforts.’’
“We believe our inclusion as a representative vendor in the Gartner Market Guide validates why the world’s best-known brands rely on SSTS, to drive their Mobile Test Automation,’’ said Avinash Tiwari, Director, Smart Software Testing Solutions, explaining the unique position of the company in the Indian market.
pCloudy is the flagship product company of SSTS (Smart Software Testing Solutions). Avinash Tiwari opines that the market in India is becoming more and more receptive of automated testing solutions and cloud based testing platforms. More and more organizations are feeling pressure to make releases more frequently. Moreover, creating right test infrastructure for mobile Apps is a key concern for organizations. pCloudy is in a unique position to serve both the challenges.
If you have a Gartner Account, you can download the report by clicking on this link.
Gartner Disclaimer
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties,expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
(1) Gartner, Inc. ‘’Market Guide for Mobile App Test Automation Tools’’ by MaritessSobejana, Adrian Leow, 25 April 2016
July 6th, 2016 by Avinash Tiwari
Testing Mobile Apps
If you are someone using a smartphone to get through most of your day, then I’m sure would you’ve experienced it – You are in an elevator using your smartphone trying to find a restaurant for dinner but the app you are using unexpectedly crashes, or you are in a crowded subway using your smartphone to upload a video to some social media portal but the app you are using takes too long to respond, or you are just waiting for a friend in the basement of a parking lot and in the meanwhile you decide to shop for some new shoes but the e-commerce app you are using does not respond the way it should. Isn’t it frustrating when the apps you are using does not work when you require it the most? In these situations, the first thing you do is simply uninstall it, or in the worst case write a bad review.
Strangely, though most of the apps are functional and performance tested, they are still found to be defective by the users. It could be possible that not all scenarios are covered during testing. So, where is the gap? Deeper investigations have revealed that mobile apps behave differently in different network environments. In the above scenarios particularly, it seems like there is more evidence indicating that these issues are caused due to irregular mobile networks in locations such as the Elevator, or the crowded Subway, or the basement of a parking lot.
Why do apps behave differently in different network environments?
The network has a huge impact on the behaviour of your apps. Most of the apps that we use today are functioning by constantly communicating and fetching information from some servers or users that are miles away. The communication that happens is on wireless networks that are unfortunately not always reliable or consistent. Though most MNOs have established wider and better networks, it is practically difficult to achieve absolute seamless connectivity with excellent signal stability using existing wireless technologies. This is because, inherently, wireless networks have several inconsistencies like the lack of bandwidth in some places, the drop in packets caused by handovers and routing while moving, and more interference from other networks. Additionally, the networks through which data is transferred wirelessly also vary based on the technology used like GPRS, UMTS, HSDPA, LTE etc.
Why should apps be tested in different network environments?
The unpredictability of wireless networks has a huge impact on the functionality, performance and user-experience of an app. This is why we often see certain functional defects, performance defects, and sometimes crashes only when the app is running on a certain network or in some specific locations. Besides, mobile apps are expected to function on everything from an excellent Wifi connection, to a snail-paced 2.5G connection. Hence mobile apps have to undergo a wide range of tests on different networks with a variety of conditions before they are released to the users.
Are you testing your apps on different network conditions?
Traditionally, mobile apps are tested using the best networks in perfect conditions. But in real environments when the apps are being used anything can go wrong – the network bandwidth might not be enough, the latency might be high, data losses might occur, or there might be interference from other networks. App development companies have now realised the impact a network can have on their apps and are slowly filling up the gaps in mobile app testing. They are now focused on ensuring the app’s functional stability and relentless performance despite network inconsistencies. But testing mobile apps on different varieties of networks with a range of uncommon conditions is overwhelming and challenging. Moreover, these apps have to be tested on numerous mobile devices. It’s a tedious but crucial task. Additionally, tools that support such testing are scarce.
How can you test your apps in different network environments?
- Testing Mobile Apps using live networks – Live networks can be used for testing mobile apps. The apps can be installed on real devices and can be tested on various networks like GPRS, UMTS, LTE, CDMA, etc. But the procedure of using real networks for app testing is slow, cumbersome, and mostly provides inaccurate results. Testers need to travel to different locations or wait for the network conditions to occur for conducting the tests. Furthermore, when the developers try to reproduce the bugs in order to fix it, the exact network conditions are hard to be replicated; eventually, most of the bugs will be closed as “Cannot Reproduce” while the bug still remains.
- Simulated networks – Network related tests for mobile apps need not be such a hassle if the testers are given full control of the network being used. Using a simulated lab network, testers can easily test their mobile apps by simply selecting and applying the type of network profile to be used. They can also create and customize different network profiles by simply changing parameters like Uplink Bandwidth, Downlink Bandwidth, Latency, Loss Percentage, Packets Corruption, etc. This setup helps testers to replicate a myriad of network conditions for testing mobile apps.
On a larger view, when a mobile device cloud like pCloudy is being used you can test your mobile apps on varieties of networks, on hundreds of real mobile devices in parallel, through manual tests or automation runs. It is one platform that takes care of your entire app testing needs by helping app developers to easily test and experience the app functionality and performance on different networks and different mobile devices.
May 23rd, 2016 by Avinash Tiwari
Introduction to iOS TestFlight
TestFlight is an online service used to distribute and test beta iOS applications. Developers can use Test Flight to evade the Apple’s bitter binding procedure for testing new versions of their apps. It provides over-the-air installation, testing, tracking and managing of Beta apps. In simple words, TestFlight acts as a link between the developer, the tester, and the Apple Developer Center.
How is it useful?
- Distribution of the app over-the- air to Testers
- Tracking and managing testing using TestFlight’s dashboard
- Receiving feedback from testers to improve the app
- To obtain reports about the device models, OS version used while testing the App
- Automatically records any crashes the testers encounter and sends it to the Developer
This tutorial will walk you through integrating TestFlight into your own apps.
What do you need to start?
You have to have your certificates, app ID, and provisioning profiles in place before distributing your apps using Test Flight. Here’s the list of items you need:
- A Developer ID to login to your developer account.
- A Certificate for Distribution to export builds and distribute apps
- An App ID
- A Device UDID
- Ad Hoc Provisioning Profile to distribute your app on testers devices
- Sign your app with a valid ad hoc provisioning profile
These items will help you obtain an .ipa file that can be distributed for testing in iOS devices with UDIDs mentioned in the provisioning profile of the app.
Uploading your app for Beta Testing
Open up your project in Xcode, make sure you have a correct Bundle Identifier and that you’ve chosen the correct Distribution Certificate:
Choose Product\Archive from the top toolbar:
Once Xcode finishes archiving your project, click the shiny blue Submit to App Store… button:
Choose your development team
Hit Submit
Wait for your build to upload. Once done you will receive a “Submission Successful message”
Inviting Testers
May 17th, 2016 by Avinash Tiwari
Espresso with pCloudy
Now, run your Espresso Scripts on hundreds of Mobile devices and build quality apps faster than ever.
Several Mobile Automation Frameworks like Robotium, Calabash, Appium , KIF & uiautomator have been developed in recent years. Among a few popular ones, Espresso is one of them.
You probably know, Espresso is a testing framework for Android apps. It is predominantly used by developers and testers for testing user interactions (UI) to ensure that users do not encounter crashes or unexpected results while using an app.
There are several benefits of using Espresso, the primary ones are:
- It automatically synchronises test actions with the UI of the app being tested. Meaning? To improve the reliability of tests, it detects when the main thread is idle and runs the test commands at the appropriate time.
- This capability also relieves you from having to add any timing workarounds, such as a sleep period in your test code.
- Espresso has an API that is small, easy to learn and built on top of the Android instrumentation framework.
- It also supports testing activities outside the app like camera, browser and dialer etc which Appium does not support.
However, when you have limited time and money it is a challenge to take full advantage of your Espresso scripts in improving the quality of your apps. Luckily, there are ways to overcome this.
Conventionally, you would spend a huge amount of money to purchase several Android devices, and spend endless hours running your scripts independently on each device. Today, thankfully you don’t have to do this anymore, there is good news. With the help of a cloud-based testing platform like pCloudy.com, you can run your scripts on hundreds on real mobile devices in parallel.
Yes, using pCloudy.com you can test your apps on any device of your choice and pay a just minimum price for the time you’ve used the device. You can not only save money, but you can also save your time by running your test scripts on multiple devices in parallel and get a detailed execution report.
Here’s how you can use Espresso on pCloudy devices:
May 6th, 2016 by Avinash Tiwari
Calabash Automation Testing
Improve the quality of your iOS and Android apps by running your Calabash Test Scripts (Calabash Automation Testing) against real phones and tablets on pCloudy.com.
Along with Appium, Robotium and Espresso, you can use Calabash as well on pCloudy.com.
Calabash is a free open source framework for mobile automation testing. It is cross platform, supporting both iOS and Android. It consists of two libraries – calabash android and calabash iOS, which helps in writing tests in domain specific languages. Calabash consists of libraries that enable the test code to interact with apps. Each of these interactions consists of a number of end user actions like gestures, assertions or screenshots. The Tests can be written in simple, self-explanatory language that can be easily followed by even non-technical people.
Further, to increase the efficiency of testing and to improve the quality of your iOS and Android Apps and you can automate your Calabash Test Scripts against a wide range of real mobile devices on pCloudy.com.
Here’s how you can use Calabash on pCloudy devices:
May 6th, 2016 by Avinash Tiwari
Test Apps on Real Devices
Using Eclipse or Android Studio to code your app? Now with just a few simple mouse clicks test apps directly and in parallel on multiple real mobile devices.
Eclipse and Android Studio are two popular IDEs for mobile app development. The reason behind their popularity is, these are open source tools and have a great community of developers whom you can turn to for any support. Anyway, if you are one of those mobile app developers using these popular IDEs then there’s good news for you!
What if you could extend the capability of your IDEs to improve the quality of your apps? What if you could test your app on hundreds of mobile devices right from your IDE? What if you could with the help of simple plugins take your mobile app development process to a whole new level?
Well, you can – with a simple yet salient solution from pCloudy.com. With a cloud based device lab, pCloudy is contributing to the cause of redefining mobile app testing by providing remarkably useful platform and plugins to test your apps on real mobile devices. As mentioned in our previous articles, it is not enough if mobile apps are tested on Emulators alone. We need to ensure that the apps are tested on Real Devices as well. It is also important to test your mobile apps on different devices based on a carefully analysed device matrix to hit maximum downloads. To continuously develop, integrate and release mobile apps in your DevOps environment such solutions are crucial.
Here’s how can our plugins benefit you
Description:
Our Plugins act as a wizard that allows you to:
- extend the ability of Eclipse to use real devices over a cloud platform
- select and install an app from a Cloud Drive or from local
- select multiple real devices from a cloud (Public, Private, On-premise) to test your app
- run test scripts in parallel on multiple devices
- perform Automation or Manual Testing
How does it work?
If you have already created Test Scripts ready for Test execution, then all you need to do is to select some real devices over the cloud platform and run the execution to test your app. To help you out with this, the Plugin generates a pseudo-code that can simply be copied into your existing test scripts using Android Studio or Eclipse. With simple changes in the test script, the app will get installed and test execution is performed on the selected mobile devices. In the end, a detailed automation report will also be generated through our platform.
With these plugins app developers and testers can:
- choose from hundreds of mobile devices to test your App
- effortlessly test their apps directly on real mobile devices
- can view or perform an activity on mobile screen from directly within on your IDE
- perform automation runs in parallel on multiple mobile devices
Download and install the plugin:
April 22nd, 2016 by Avinash Tiwari
pBot
The future of app testing in IoT lies in integrating app interaction with external hardware triggers. While devices such as phones, tablets and wearables may already seem ubiquitous, a networked array of entrenched systems and sensors commonly called the Internet of Things, could eventually race past all of them in its worldwide reach and number of applications and services.
Connectivity through the IoT is reliant on many different wireless standards. For example, to do anything at all, an IoT-enabled device might have to interface with Wi-Fi, Bluetooth, amongst others channels. For software makers, these potential issues with connectivity and infrastructure will inevitably shape the design of their applications.
Speaking of wireless connectivity technologies, NFC applications can now be tested with pCloudy’s new testing solution- pBot. Before explaining pBot, let’s talk about what Near Field Communication is all about.
NFC is a set of short-range wireless connectivity technologies that are perfect for transmitting small amounts of data with minimal setup time and power consumption. NFC applications include contactless transactions, data exchange and simplified setup of more complex technologies. NFC-enabled devices typically support three operating modes which are Reader/Writer, Peer to Peer and Card Emulation. This fuels contactless payments, exchange of business cards or digital pictures.
As the number of available NFC-enabled mobile phones and tablets increases, the market will see a growth in applications such as mobile payments, ticketing, smart posters, as well as access control, data sharing and additional services.
pCloudy’s pBot has made NFC testing flawless with its advanced functionalities. For instance, NFC tagged visiting card. The card data i.e., the contact information is being read by an app on mobile device. Here, pCloudy mobile app interface integrates robotic actions which triggers the robot to perform NFC action.
Smart Payment Terminal Testing with pCloudy pBot
With pBot, smart payments are now made realistic through testing payment terminal for EMV certification. For instance, when a credit card swipe device needs to be tested with real credit cards, the pCloudy mobile app interface integrates robotic actions which triggers the robot to action on payment terminal.
Thus, app testing with pCloudy’s pBot is now made futuristic with distinctive robotic functions. App testing should include IoT specific test strategies, support SDLC under context, and have access to on-demand test teams with requisite specializations. Success depends on the ability to influence a dedicated infrastructure, cloud capabilities and a wide array of test tools.
As enterprises leverage smart systems that ‘sense’ the environment and execute programmed actions, IoT driven testing is crucial to strengthen IT initiatives with security, intelligence and flexibility.
April 15th, 2016 by Avinash Tiwari
Trends of 2016
2015 saw a great revolution in the mobile app testing industry with many new breakthroughs being achieved. The year witnessed an emphasis on modern mobile applications being compatible with various devices. However, modern mobile apps also need to deliver personalized and flawless user experience. This year there will be a huge advancement in the mobile application testing space with latest versions of mobile operating systems and rapid technological transformations. We list the top seven mobile app testing trends for this year.
Rise of cloud based App certification platforms
Mobile Device Clouds have become an integral part of Mobile App Dev cycle for many organizations. These services help developers and testers by providing access to a wide range of smartphones and tablets. This trends will pick up further this year with many new players entering this space. These services are becoming available at a very affordable price for customers.
We see a new trend where these device cloud platforms will transition to full fledged certification platforms. Customers can get their Apps tested on multiple devices on parameters like behavior, performance, security etc. in completely automated way. All this using single click.
Agile and continuous delivery
Mobile Apps are becoming backbone of business for many organizations. In fact, trend of App only businesses are increasing rapidly. In such situations traditional models of development are a complete misfit. Delivery cycles are shorter and you can’t wait long to fix a customer feedback.
Agile Dev cycles, continuous delivery and deployments are becoming key to success of any Mobile App strategy. We see an increase in this trend this year as well.
Role of Test Automation
Continuing on earlier point, Agile Dev cycles, continuous delivery and deployments are the new necessities for success of Mobile Apps.
The only way to succeed with above approach is to automate as much as possible. We see a rise in trend where organizations are effectively defining their tests pyramids. As the pyramids suggests maximum automation should happen at Unit tests level followed by API level tests. Scope of monkey and UI tests are driven by stability of App UI.
Growth of Open Source
We have seen rise of open source tools in last couple of years. Test Automation tools like Selenium and Appium are giving commercials tools a run for their Money. As the maturity and community support for open source tools are increasing, more and more organizations are adopting open source tools.
Improved Focus on Security
The requirement for comprehensive security and penetration testing is set to increase exponentially in 2016. People store sensitive data on their mobile phones like bank details, credit card details and other personal data which can be easily used for carrying out financial thefts. Thus, testers need to focus on renewed security by implementing comprehensive security.
Internet of Things
With convergence of cloud and IOT, we see a rise in consumer IOT Applications.
These applications require testing solution that go beyond traditional methodologies and integrate Software validation with smart hardware triggers and touch of Human Experience.
Recently, lot of interesting developments like Google launching project Brillo – which is based on Android – Gives us hope that industry will move towards standardization and this will provide a big boost to development of IOT validation systems.
January 13th, 2016 by Avinash Tiwari
Simplest App Testing Platform
With the advent of smartphones into our lives, “Mobile First”, “Mobile Only” and “Enterprise Mobility” has become new buzz words. However, for any Mobile program or an app to be successful, it takes careful planning and execution of testing.
Traditional app testing is time consuming because it requires a huge, dedicated infrastructure and resources that is to be used periodically. It is safe to say that Mobile Device Cloud is the next big thing. Working on Cloud opens up new panoramas of opportunity for testing.
This is where pCloudy comes into the picture.
pCloudy is a mobile application testing platform designed to increase your App Testing coverage while saving significant time and cost. All this with a simplicity you wouldn’t believe until you see it yourself. So, what makes pCloudy the simplest app testing platform?
‘’No requirement of additional softwares’’
Test with Real Devices (yes, it’s all real devices) through Public Cloud – Increasing test coverage was never so easy
pCloudy allows users to test on a wide-ranging list of real iOS and Android mobile devices and tablets. Now, you can ensure that your App is compatible with a variety of devices. You can also replicate and fix your production bugs easily.
Single click access to real devices – Super easy
Firstly, accessing their cloud device is made really simple. Access to real devices is a single click away using any Html5 browser. With security being a key factor, pCloudy makes sure information is protected and is secure over web sockets.
Single click access to testing tools – Save time
pCloudy provides single click access to tools like “Screenshot”, “Live Logs”, “Crash Reports”, “Performance Metrics”, “Videos” and “Geo-location”. Imagine how much time do you spend performing these steps on each device.
Any time parallel execution of Automation scripts – Save time and Increase coverage
Creation of Automation scripts for Mobile Apps is easy. What’s challenging is “How to run the scripts in parallel on real devices during regression run”? “How to create consolidated reports with Device level stats built-in?
Whatever is your Automation framework (over Appium, Calabash or Robotium), you can focus on script creation and leave the rest to pCloudy. You can also run it directly using your build integration system like Jenkins
pCloudy allows automation run on any no. of devices in parallel. (Yes, truly in parallel). Imagine the coverage you can achieve and how much time can you save?
Increase efficiency of your lab
One of the most novel features of pCloudy is that they offer an “in-premise” setup at your organization. To make a demanding test schedule for an organization simpler, pCloudy provides the essential hardware and give you the liberty to set up your own secure device lab. With this, you will have complete control of your own on-premise lab with multiple benefits like plug-in play, day zero installation, no limitation on the number of concurrent logins, and a smooth mobile device management where you can monitor devices closely. There is no limitation of number of devices which can be attached to an on-premise setup. Apart from this, you can increase the USB hub to the main setup easily as per your requirement.
‘’Unlimited numbers of devices can be attached to an in-premise setup’’
January 5th, 2016 by Nilesh Tarale
Mobile App Testing Tips for IOS 9.x
iOS 9 and later provide users with the ability to enter power-saving modes.
As per Apple IOS 9.0 update ( source : https://developer.apple.com/library/ios/documentation/Performance/Conceptual/EnergyGuide-iOS/LowPowerMode.html)
“Users who wish to prolong their iPhone’s battery life can enable Low Power Mode under Settings > Battery.
Once enabled, Low Power Mode changes the battery indicator in the top right corner to yellow, reminding users that their device is running in a more efficient state.
In Low Power Mode, iOS conserves battery life by enacting certain energy-saving measures. For example, the system may:
- Reduce CPU and GPU performance
- Pause discretionary and background activities, including networking
- Reduce screen brightness
- Reduce the timeout for auto-locking the device
- Disable Mail fetch
- Disable motion effects
- Disable animated wallpapers
The mode automatically disables when the battery level rises to a sufficient level again.
Impact on Apps
Your app should take additional steps to help the system save energy when Low Power Mode is active. For example, your app could reduce the use of animations, lower frame rates, stop location updates, disable syncs and backups, and so on.
What if your App is resource intensive, and user is trying to use the App when iPhone is in the low power mode? You App might crash. Regardless of the scenario, You need to make sure your app works and the best way to do that is to conduct mobile app testing on real devices. You need to put your App through rigorous testing on low power scenario on real devices.
