Organizations have been using load testing for years to understand how their Web applications will perform when stressed. Mobile apps present a different testing challenge, according to Amir Rozenberg, a senior product manager at mobile application testing company Perfecto Mobile.
“It’s about the expectation of the end user,” he said. “The app has to perform in the wild now. In fact, performance is now more important than the app itself.” He cited a study in Capgemini’s World Quality Report that found 67% of enterprises today rate performance as their top priority for mobile testing activity.
But, Rozenberg said, there is a gap between testing in clean-room environments and in the real world. What would happen to performance, the company wondered, in different network conditions, or in a lot of load hitting a back end because of an ongoing sporting event?
So Perfecto Mobile, which uses the actual mobile devices to test upon, has created a solution—MobileCloud Performance—that can simulate conditions a user might encounter in the real world, such as entering an area with limited cellular coverage. “Our objective is to take the device off the shelf, don’t jailbreak it, and install the app as an end user would, and access it as if we’re users,” Rozenberg said.
“In combination with our mobile cloud, we connect to network virtualization that can simulate conditions in Manhattan, for example. From the UI, we measure availability and the performance of the app. We can show metrics to portray a complete picture of bottlenecks to optimize the experience.”
Scripts facilitate this automation, Rozenberg explained. “We can add different network conditions. We can simulate someone traveling and going into a tunnel, or what does the app look like in Edge versus WiFi versus 4G,” he said. Further, he noted, different devices will behave differently under the same network conditions.
The company’s solution, he said, combines real devices and network virtualization with load simulation in one common script. “There are solutions with emulation, but they don’t show resource constraints, or they make developers put something into an app, which I find highly objectionable,” he said.
After the test, the solution provides a script execution report that shows whether or not the test was a success, shows timer results and can create a video showing the complete execution of the application during the test, Rozenberg said.
MobileCloud Performance components
Access: MobileCloud platform for access to a wide range of real devices
Scripting: MobileCloud Automation or HP’s UFT Mobile
Network virtualization: integrated component for simulation of network conditions and analysis of network traffic
Load generation: HP LoadRunner and MobileCloud enable measurement of mobile user experience while the server is presented with load conditions. Mobile synthetic loads can be created based on previously recorded network sniffer files.
