Cross browser comparison (Focus on iOS)

Hi

It is a fact that more and more hybrid/web application are being developed lately, HTML5 applications and more

The common assumption by the application developers is that since it is a web application is will run cross platform without too much effort/QA and UI activities.

It might be true in some cases (simple ones), however on top of the complications in testing web application, we must not forget that each app need to be compliant to iOS and Android UI guidelines (icons, fonts etc.), and also important point to keep in mind is – The cross browser compatibility.

Each user (Android or iOS and soon Windows Phone) may chose the prefered browser which he likes to surf through and use. THat user will not change his browser and will expect that “your” application will run top-notch on his preferred browser.

In this post i will not cover all existing iOS browsers (nor android), but the top leading ones – My recommendation is to prepare similar matrix of testing for the existing web browsers for mobile and perform at least some level of sanity on each to assure your application works properly and also meets the desired guidelines.

For iOS we are familiar with the following browsers (The below iTunes URLs for downloading each:

– Safari (THe most common and default browser)

– Chrome (Google) – http://itunes.apple.com/us/app/chrome/id535886823?mt=8

– Dolphin – http://itunes.apple.com/us/app/dolphin-browser/id452204407?mt=8

– Opera Mini – http://itunes.apple.com/us/app/opera-mini-web-browser/id363729560?mt=8

– Mercury – http://itunes.apple.com/us/app/mercury-web-browser-most-advanced/id331012646?mt=8

Please see below some screenshots of the exact same web page (BBC News) which is a high quality web site being run on the above browsers (Not too much difference which is good news :), however there are some L&F differences – for other apps i am sure that the situation will be different)

Safari Browser:

Google Chrome:

Mercury Browser:

Dolphin Browser:

Opera Mini:

Not to forget that the above if appears quite different from one browser to the other will introduce additional challenges for the automation team.

To sum up, mobile known matrix of devices and os will also extend toward mobile

Browsers per platform, a vital thing to cover which will also complicate automation.

Good Luck

Regards,

Eran

Picking the right handsets for your project

Hi

We all know that the mobile world is dynamic, plenty of new handsets are being shipped at the same time in which we develop our product and testing on the (what we believe) is the “hottest” handsets in the market.

It is clear that being agile and fast in the way we develop, test and deploy our mobile products is a key to be attractive in the market, however it is also impossible to support all handsets and be ahead of the market.

So – The way to be up to date in the offering, is not simple but possible.

When you start developing your product keep in mind that by picking the “right” 10 handsets which are “hot” in the market you can reach the coverage of ~50% of the market (Note that there are lead devices which represent a whole family of handsets and can give you a lot of value by testing on it), as well if you go to ~30 devices you may reach up to ~80% coverage of the market.

How should you decide than?

The way to do the picking of handsets should combine the 2 following aspects:

– Market research

– Right family identification/lead devices

Market Research: The way to determine what is relevant in the market is to do some research and analysis – either through leading mobile blogs, or even simple – going through the leading mobile operators in the world, and seeing what they are currently selling (e.g. Vodafone global lists today in the top list of devices in Germany: Samsung Galaxy SIII, Samsung Galaxy SII, SEMC Xperia Arc S etc. – http://shop.vodafone.de/Shop/smartphones/, if you go to Vodafone UK you will see mostly the same ones, as well as HTC One X and others http://www.vodafone.co.uk/brands/android/index.htm)

Doing a matrix and unification of handsets between the world leading carriers in Europe/U.S/Asia should give you the lead handsets which you would like to support and test in the 3-6 months ahead (Per OS – Android, iOS, Windows Phone and BlackBerry).

Families: The aspect of family should be a subset of the above list If e.g you reached a common list of lets say 50 handsets, i am mostly certain that the list can be cut into half by doing proper comparison between the various handsets by their OS, Screen resolution and OEM (This can be done through sites like GSM Arena – http://www.gsmarena.com)  and minimizing the list by leads, members and families.

Please find attached to the post an up to date list of common handsets by OEM which is sold world wide these days to ease your pain 🙂  –> As you will see, there are a lot of similar handsets across all large operators which can show the main devices to focos on.

MobileWorldHandsetsDistribution

P.S: With regards to the leading Android/iOS tablets these days:

iOS – iPad 2 and iPad 3

Android – Samsung Galaxy Tab 10.1, Motorola Xoom, Asus Nexus 7, Dell Streak 7, Samsung Galaxy Tab 7, Sony Tablet S. Asus Transformer TF300, Asus Transformer TF700

Regards,

Eran

Cross Platform mobile development tool – Titanium

All

This is a Testing blog however i recently get a lot of questions around Mobile development/porting for cross platform projects (Android/iOS etc.).

For this cases there are several good and free tools out there which can be used such as PhoneGap (http://phonegap.com/download/), EggPlant (http://www.testplant.com/products/eggplant/for-cross-platform-testing/) and Titanium.

In this short post i will just point you out to the tool and give a very high level details about it for you guys to go and try it out for yourself

Titanium is a free tool by Appcelerator (http://www.appcelerator.com/platform/titanium-sdk) which allows quite quickly to develop a cross platform mobile application which can be than deployed on iOS and Android Phones/Emulators and also PC Web browsers.

The tool allows to develop in Java Script and customize your resources pending the operating system you which to deploy on:

You need to have on your machine the Android SDK and point its location in the Titanium SDK to allow execution and debugging on the Android Emulator.

You have through the SDK options to create new project from samples or templates (HTML5, Tabbed APP and more).

Feel free to give it a try and comment

Regards,

Eran Kinsbruner

Mobile project planning

Hi

Up until now i covered in the various posts the aspects of Mobile testing, trends, tools for automation, mobile platforms etc.

All of the above does not help much if the mobile project is not well planned in advanced with SMART milestones, efficient choices of devices (lead/gold devices), right tools and Agile ALM (Application life-cycle management).

In this short post i will advise the most important project milestones and how to define them.

First milestone:

Product definition and feature list –> In this milestone the product management ought to chose the supported platforms and devices for the project (based on thorough market research world wide and based on the projected release date which should be relevant for the selected devices), the project scope (the feature set should be well defined, the supported languages/localizations should be defined, the required certifications (Google CTS, Google Market, Apple App Store certification and more) should be clear, the target customers and of course the projected release date.

Not to forget the UI/UE aspect which is a critical aspect in any mobile project (Not only because the OS requirements but also for the end users).

Second Milestone:

– Core team build up and preliminary work plan implementation, staffing etc.

– Purchasing/ordering of equipment as needed, tools for development and automation should be brought up for approval

– High level Test Plan and Product specification should be developed during that milestone and approve by the core team

Third Milestone:

– Product development

– Unit test development

– Test development on the lead/gold devices per platform (Manual and Automated)

– Documentation creation (As needed and as defined by product)

– Sanity testing on supported devices, emulators, and cloud based handsets.

fourth Milestone:

– Feature complete

– Test development complete

– Full cycle execution on lead devices

– Porting kick off –> Building of subset devices (Either from the lead devices family or new families) versions for testing (Only sanity testing is required for devices within the lead devices family, while full cycles on new family members)

** Family can be defined by a group of devices from the same OEM with the same OS and Screen resolution (e.g. Samsung Galaxy S and Samsung galaxy S Plus are related to the same family –> http://www.gsmarena.com/samsung_i9001_galaxy_s_plus-3908.php ; http://www.gsmarena.com/samsung_i9000_galaxy_s-3115.php).

Fifth Milestone:

– In this stage the project should be well tested, debugged, ported to at least several devices and families, as well as documentation should be ready, certification status should be GREEN, and the bug trends for the project should also be minimal, quality criteria is mostly met.

– Project release to the various markets for approval is being done

– Based on approval and if needed pre-customer approval the project is released!!

The above is just in a nutshell list of items which i feel are a must in a common project plan for a mobile project, i might have missed few items, or the order can be different in few companies, but the above should bring you to a safe release if followed properly.

Thanks,

Eran Kinsbruner

Free iOS automation testing tool – Test Studio for iOS

Hi

I will not write too much in this post, just wanted to provide a link for a free mobile automation tool for iOS web/GUI applications which is available in the App Store

http://www.telerik.com/automated-testing-tools/ios-testing.aspx?utm_source=twitter&utm_medium=sm&utm_content=FRDTNCKNWJ&utm_campaign=ad

The tool provides a simple record/replay mechanism on the iOS native platform and fits basic and not too complex web and native applications.

Soon it should allow sync of the test results and more in the cloud.

Test Studio for iOS does not use image based element detection. It relies on object based recording instead.

I recommend to give it a try

Regards,

Eran Kinsbruner

Integrating a new Android AVD (Emulator) into Android SDK

Hi

In this post i would simply show the quick and simple steps on how to add a new OEM Emulator into your Android SDK and run a sample Android application (APK) on it or on a specific Emulator.

The assumption for this post is that you’ve already downloaded and installed on your Windows/Linux machine the latest Android SDK.
If you have not yet done so, you may download the SDK from: http://developer.android.com/sdk/index.html

** Please note that the SDK requires a JDK (Java Development Kit) to be installed on your PC.

Once you have the SDK installed, you will have a basic skin/AVD (Android Virtual Device) to work with, which comes with the SDK.

As application developer/tester you know that 1 emulator will not give you enough confidence about your application functionality and behavior (without getting into the debate of the use of real device – which i tend to recommend to start ASAP and in the worse case, no later than Beta milestone), so you need to get as much coverage as possible using as many emulators as possible.

Most OEM’s develop and provide for free many emulators which you can simply add to your sdk (We have Samsung, Motorola, LG and HTC as well as Sony Ericsson who provide Tablets and Smartphones emulators which you can download and deploy in the SDK).

From your SDK installation path, simply paste the ZIP with the AVD files into the folder:

{SDK_HOME}/platforms/android-xx/skins –> When android-xx stands for the version of Android on which you wish you run your application on (e.g. – Android-10 stands for Android 2.3.3 Ginger Bread, while android-15 stands for Android ICS 4.0.3).

See example of a folder into which i created few simulators (or AVD’s):

The problems which people often ran into or ask about are, what is the method to install and run an application through the ADB command line on a specific emulator while i have 2 or more running?

The answer is simple:

From the command line assuming you have few AVD’s running, run the command “adb devices”

You will get an output in the below form:

“C:\android-sdk_r18-windows\android-sdk-windows\platform-tools>adb devices
List of devices attached
emulator-5560   device
emulator-5564   device”

If you than want to install an APK on 1 of the above emulators (e.g. emulator-5560), simply run this command:

“adb -s emulator-5560 install “demo.apk”

 

That’s it

I know that it is a quick and short post, if you have any problems or questions about the above – feel free to ask

Regards,

Eran Kinsbruner

JAMO Solutions – Mobile test automation tool

Hi,

I am sure that most mobile testers heard about JAMO solutions, but if not than i will briefly introduce this solution.

In past blogs i mentioned few powerful automation tools such as PerfectoMobile, Keynote Device anywhere, SeeTest by ExperiTest and FoneMonkey (MonkeyTalk).

Except of the Monkey Talk – all others are commercial tools and not free.

Jamo is also a commercial and not free solutions which comes in various licensing models from a company based in Belgium  (http://www.jamosolutions.com/editions.html).

The solutions is based on a plug in to various of IDE’s such as QTP (HP), Eclipse, and Visual Studio, which than allows connectivity over USB or WiFi through a Device manager to the mobile platform (iOS, Android, BlackBerry and Windows Mobile) on which an agent is also installed to provide the full control and automation via record/replay, easy to input text utils etc.

For Android OS device automation using eclipse, you can see a short demo of the tool and its methodology:


In the same manner as the video above users can automate their scenario’s through QTP or Visual studio for the various OS platforms.

For more information or inquiries please contact Jamo through their web site:

http://www.jamosolutions.com/home.html

Regards,

Eran

Mobile testing using Benchmarks

Hi,

We know that many devices vary between each other in their HW capabilities, meaning – the fact that 2 devices by 2 different or similar OEM’s, running the same OS/different OS version will score differently in a benchmark run or a performance test.

This is usually due to hardware differences, defects related to memory allocation, and others.

The use of mobile benchmarks had become quite significant and useful and can allow the application developers and testers to do objective testing on many different devices using a benchmark tool and get unique scores.

By comparing the overall score among the devices under test they can reach a conclusion which device has the higher performance and decide to either use it as “Gold” device for its porting platform, or if the score is extremely low, use it for testing when that device serves as the low performer in the list allowing the team to gain some prerequisite information on how their application would run on the device/s.

The benchmarks are usually downloaded for free from the Android or App store markets, and allows at the end of the quick test run to compare the device score with other devices which ran in the past and see the overall position of the device under test in compare to the market.

Usually the tests involve mathematical calculations (such as finding the prime number from a list), string sorting or number sorting, 2D/3D gaming with FPS calculations, reading and writing to a memory of the device (SD Card etc.), and some other CPU tests.

I strongly recommend to use such tools as part of the testing and as part of the decision for porting a new/existing device.

Information on commonly used benchmarks is available in the web, however i have good experience with the following ones:

– For Android – PassMark and Antutu (Which also provides Battery saver tool and CPU master to control the CPU speed)

http://www.androidbenchmark.net/passmark_chart.html

http://www.antutulabs.com/AnTuTu-Benchmark

 

– For iOS – PassMark and iBench

http://itunes.apple.com/us/app/ibench/id385703033?mt=8

Good Luck

Regards,

Eran Kinsbruner

Latest Mobile world news (July 2012)

Hi,

I’ve collected few topics which are “hot” these days in the mobile world and which i find interesting and relevant to the global audience.

  • Apple states that NFC support  (Near Field communication) might be included in the next iPhone 5 device

http://www.businessinsider.com/iphone-5-may-include-nfc-chip-for-mobile-payments-2012-6

  • RIM (research in motion) is considering selling its business to Facebook or Amazon (Treat this as a rumor rather than a fact)

http://www.businessinsider.com/rim-considers-selling-its-handset-business-2012-6

  • Time spent on mobile apps vs. web browsing is growing year by year – see below latest trends (Minutes per day):

  • Microsoft announces on the new Windows 8 OS with Internet Explorer 10 release with new features, enhancements and more, as well as new Metro customizable UI

http://windows.microsoft.com/en-US/windows-8/release-preview

  • Check out the list of 20 best mobile phones in the world according to techradar site:

http://www.techradar.com/news/phone-and-communications/mobile-phones/20-best-mobile-phones-in-the-world-today-645440?artc_pg=22

Enjoy 🙂

Regards,

Eran Kinsbruner

Reference for Google Jelly Beans 4.1 Google now

Hi,

In this quick post, i just want to atract your attention to one of the coolest features which is coming to the new introduced version of Google – JeakkyBeans 4.1 and called ‘Google Now’ – Voice command controller and more (Provides guide based on your location, and additional information based on your social activity)

At first, check out this short video for overview of the feature:

You may also read more about the first JellyBeans tablet which was also introduced lately and will be the first to come with this new OS – Nexus 7 by ASUS

http://www.wired.com/gadgetlab/2012/06/hands-on-google-asus-nexus-7-tablet-impresses/

Regards,
Eran Kinsbruner

Create a website or blog at WordPress.com

Up ↑