Search This Blog

Monday, 16 March 2015

Some Open Source Mobile Automation Tools

Some Open Source Mobile Automation Tools

  • MonkeyTalk :-
Platforms Supported :- iOS and Android
Scripting Language:- JavaScript
URl :- http://www.gorillalogic.com/

  • Appium :-
Platforms Supported :- iOS and Android
Scripting Language:- Web Driver and Json wire Protocol
URl :- http://appium.io/
  • Robotium :-
Platforms Supported :- Android only
Scripting Language:- Java
 URl :- http://www.gorillalogic.com/
  • Calabash :-
Platforms Supported :- iOS and Android
Scripting Language:- Cucumber(Ruby)
  URl :- http://calaba.sh
  • Sikuli :-
 Platforms Supported :- iOS and Android
Scripting Language:- Python(others to come)
URl :- http://www.sikuli.org/
  • Native Driver  :-
 Platforms Supported :- Windows, iOS and Android
Scripting Language:- Web Driver API
  • Testing With Frank  :-
 Platforms Supported :- iOS only
Scripting Language:- Cucumber(Ruby)
  • Selendroid :-
 Platforms Supported :- Android only
Scripting Language:- Selenium 2 Client API
URl :- http://selendroid.io/

Thursday, 12 March 2015

Must Asked Interview Questions for Mobile Tester

Difference between Simulator and Emulator?

Simulator and Emulator both is are virtual device given by iOS and Android to test or develop the application on virtual devices. Below are the major differences in more simple words to know what is Simulator and Emulator:

EMULATOR :-
  1. It comes with the eclipse setup to test or develop the Android Application on virtual device.
  2. In Emulator we can change the hardware configuration as per as our need, for eg;-We can set the RAM,Size and Storage capacity of the virtual devices.
  3. We can modify the name of the Emulator.
  4. We can create Emulator for various Android Tablet and Phone.

 SIMULATOR :-
  1.  It comes with the X-Code setup to test or develop the iOS Application on virtual device.
  2.  In Simulator we can't change the hardware configuration as per as our need. It always comes with predefined Hardware configuration.
  3. We can't modify the name of the Simulator.
  4. Simulator gives virtual devices for iPAD and iPhones. 
  5.  
 Difference between Jailbreaking, Rooting and Unlocking?

Jailbreaking :-
  1. Jailbreaking is the process of removing the limitations put in place by a device’s manufacturer. 
  2. Jailbreaking is generally performed on Apple iOS devices, such as the iPhone or iPad.
  3. Jalibreaking removes the restrictions Apple puts in place, allowing you to install third party software from outside the Apps Store.
Some people may have the perception that jailbreaking is only used for piracy, but this is not the case,  jailbreaking allows you to do things like change your iPhone’s default browser and mail client. Essentially, jailbreaking allows you to use software that Apple doesn’t approve.
For example, there is now a Microsoft Surface RT jailbreak that allows you to install unapproved desktop programs. (By default,  Windows RT systems only allow you to run desktop applications written by Microsoft.) However, the desktop apps must be compiled for ARM, so you can’t run any Windows desktop programs you already have, although open-source applications could be tweaked and recompiled for the Windows desktop on ARM.

Companies like Apple and Microsoft don’t want you jailbreaking to get past a device’s limitations, so you can change the default programs on iOS or run third-party desktop applications on Windows RT. To perform a jailbreak, someone has to find a security vulnerability that allows them to “exploit” the device and get around the manufacturer safeguards.
Android allows users to install third-party applications from outside Google’s app store out-of-the-box and doesn’t need to be jail broken.

Rooting :-

Rooting is the process of gaining “root access” to a device. This is generally performed on Android devices, but rooting can also occur on other devices based on Linux, such as Nokia’s now-retired Symbian operating system.
On Linux and other UNIX-like operating systems, the root user is essentially the same as the Administrator user on Windows. After rooting, you can grant specific applications access to root permissions, allowing them to do almost anything they want to the operating system. For example, an application with root permissions could uninstall system applications, install low-level system binaries, revoke permissions installed apps require, and do other crazy things. Almost anything you can do on a proper Linux system, you can do with root access on your phone.
Rooting gets around Android’s security architecture and could potentially cause problems if users don’t know what they’re doing, so Android doesn't come rooted.
On some devices, rooting may need to be accomplished via a security exploit. Just like jailbreaking, manufacturers generally don’t want you rooting. On some devices, such as Nexus devices (which are also intended for developers), rooting does not require a security vulnerability.

Unlocking :-

Many phones, particularly phones that come subsidized with a contract, come “locked” to a specific carrier. The phone is set up so that it can only be used on that carrier’s network. If you insert a SIM card from a competing carrier into the phone, you will see a message indicating that the phone is locked and cannot be used with the SIM card.
Unlocking a phone allows you to use it with a different SIM card — either to use a different carrier while travelling or to take your current phone with you while switching to a new service provider.
You will generally need an unlock code to unlock the phone. Many carriers will unlock phones once your contract is up, while phones bought outright without a contract may not be locked to a carrier at all. There are ways to unlock phones without a carrier’s permission, too.

Tuesday, 3 March 2015

Types of Mobile Testing for Mobile Apps

Types of Testing used for Mobile Apps Testing

Testing mobile apps is complex - we all know that. There are numerous platforms and OS's (think fragmentation), numerous devices, numerous types of testing scenarios and a variety of network connections and carriers.
                             You have to test across several permutations and combinations to get a zero-defect state. At each stage of the mobile quality cycle, there are specific questions that an enterprise needs to deal with.

 Here is the list and type of testing that QA needs to perform while testing any mobile Apps :-

  • Functional
    • Validation of Functionality
    • Smoke / Regressions Testing
    • Off line access testing
    • Negative Testing
  • Non Functional 
    • Network Strength/Outage/Recovery 
    • Different NW Types
    • Peripheral Testing
  • Interrupt
    • Voice / SMS interrupts 
    • Notifications
    • Battery /Cable Removal
  • Memory Leak
    • Memory Usage 
    • Memory Leaks
    • Garbage Collection
  • App Performance Testing
    • CPU Usage testing 
    • Network Usage
    • Page Render time or activity Render time
  • Usability Testing
    • User Experience
    • Competitive Analysis
    • Expert Review 
  • Installation Testing
    • New App Install
    • Uninstall and Reinstall
    • Upgrade testing

  • Vulnerability Testing
    • OWASP Vulnerabilities
    • Dynamic Testing
    • Static Code Analysis
    • Data Encryption
       
  • Language Testing
    • Validation for Locales 
    • Images and Text
    • Currencies, time zones etc.
    • Context
  •  Security Testing
    • Security Auditing
    • Vulnerability – Scanning
    • Security Scanning
    • Penetrating Testing

  • Memory Testing
    • Review of cache
    • Data storage mechanism


Basic Concept for Mobile Application Testing

Overview of Mobile Application Testing

  • Mobile applications are booming,so mobile apps are big business, but with this hyper-growth comes the challenge of releasing apps that are stable on multiple portable devices and platforms. Unfortunately, when it comes to mobile apps, quality assurance can be a complex, time-consuming and costly task. This can be especially true if you rely only on manual testing or on traditional tools for testing.
  • Mobile Application testing is a process in which Mobile applications are
    tested for its functionality,usability and consistency.
  • Mobile devices differs from screen sizes,input methods with different hardware capabilities.There are different Mobile Operating Systems in the market. The major ones are Android, IOS, BREW, BREWMP, Symbian, Windows 7.5 Mango, and Blackberry (RIM). Each operating system has its own limitations. Testing a single application across multiple devices running on the same platform and every platform poses a unique challenge for
    testers.
  • There are over 400 mobile network operators in the world;out of which some are CDMA(Code division multiple access), some GSM(Global system for Mobile communications). Each network operator uses a different kind network infrastructure and this limits the flow of information.
  • As devices differ in keystrokes, input methods, menu structure and display properties a single script doesn’t function on every device.



 Types of Mobile Application Testing
  •  Functional Testing: Functional testing ensures that the application is working as per the requirements. 
    • Type of application based upon the business functionality usages (banking, gaming, social or business)
    • Target audience type (consumer, enterprise, education)
    • Distribution channel which is used to spread the application (e.g. Apple App Store, Google play, direct distribution) 
The most fundamental test scenarios in the functional testing can be considered as :
    • To validate whether all the required mandatory fields are working as required.
    • To validate that the mandatory fields are displayed in the screen in a distinctive way than the non-mandatory fields.
    • To validate whether the application works as per as requirement whenever the application starts/stops.
    • To validate whether the application goes into minimized mode whenever there is an incoming phone call. In order to validate the same we need to use a second phone, to call the device.
    • To validate whether the phone is able to store, process and receive SMS whenever the app is running. In order to validate the same we need to use a second phone to send sms to the device which is being tested and where the application under test is currently running.
    • To validate that the device is able to perform required multitasking requirements whenever it is necessary to do so.
    • To validate that the application allows necessary social network options such as sharing, posting and navigation etc.
    • To validate that the application supports any payment gateway transaction such as Visa, Mastercard, Paypal etc as required by the application.
    • To validate that the page scrolling scenarios are being enabled in the application as necessary.
    • To validate that the navigation between relevant modules in the application are as per the requirement.
    • To validate that the truncation errors are absolutely to an affordable limit.
    • To validate that the user receives an appropriate error message like “Network error. Please try after some time” whenever there is any network error.
    • To validate that the installed application enables other applications to perform satisfactorily, and it does not eat into the memory of the other applications.
    • To validate that the application resumes at the last operation in case of a hard reboot or system crash.
    • To validate whether the installation of the application can be done smoothly provided the user has the necessary resources and it does not lead to any significant errors.
    • To validate that the application performs auto start facility according to the requirements.
    • To validate whether the application performs according to the requirement in all versions of Mobile that is 2g, 3g and 4g.
    • To perform regression testing to uncover new software bugs in existing areas of a system after changes have been made to them. Also rerun previously performed tests to determine that the program behavior has not changed due to the changes.
    • To validate whether the application provides an available user guide for those who are not familiar to the app
  •  Performance Testing: This testing process is undertaken to check the performance and behavior of the application under certain conditions such as low battery, bad network coverage, low available memory, simultaneous access to application’s server by several users and other conditions. Performance is checked on both client and server side.

    The general test scenarios for performance testing in a Mobile application are:
    • To determine whether the application performs as per the requirement under different load conditions.
    • To determine whether the current network coverage is able to support the application at peak, average and minimum user levels.
    • To determine whether the existing client-server configuration setup provides the required optimum performance level.
    • To identify the various application and infrastructure bottlenecks which prevent the application to perform at the required acceptability levels.
    • To validate whether the response time of the application is as per as the requirements.
    • To evaluate product and/or hardware to determine if it can handle projected load volumes.
    • To evaluate whether the battery life can support the application to perform under projected load volumes.
    • To validate application performance when network is changed to WIFI from 2G/3G or vice versa.
    • To validate each of the required the CPU cycle is optimization
    • To validate that the battery consumption, memory leaks, resources like GPS, Camera performance is well within required guidelines.
    • To validate the application longevity whenever the user load is rigorous.
    • To validate the network performance while moving around with the device.
    • To validate the application performance when only intermittent phases of connectivity is required.
  •  Security testing:
    The fundamental objective of security testing is to ensure that the application’s data and networking security requirements are met as per guidelines.
    The following are the most crucial areas for checking the security of Mobile applications.
    • To validate that the application is able to withstand any brute force attack which is an automated process of trial and error used to guess a person’s username, password or credit-card number.
    • To validate whether an application is not permitting an attacker to access sensitive content or functionality without proper authentication.
    • To validate that the application has a strong password protection system and it does not permit an attacker to obtain, change or recover another user’s password.
    • To validate that the application does not suffer from insufficient session expiration.
    • To identify the dynamic dependencies and take measures to prevent any attacker for accessing these vulnerabilities.
    • To prevent from SQL injection related attacks.
    • To identify and recover from any unmanaged code scenarios.
    • To ensure whether the certificates are validated, does the application implement Certificate Pinning or not.
    • To protect the application and the network from the denial of service attacks.
    • To analyze the data storage and data validation requirements.
    • To enable the session management for preventing unauthorized users to access unsolicited information.
    • To check if any cryptography code is broken and ensure that it is repaired.
    • To validate whether the business logic implementation is secured and not vulnerable to any attack from outside.
    • To analyze file system interactions, determine any vulnerability and correct these problems.
    • To validate the protocol handlers for example trying to reconfigure the default landing page for the application using a malicious iframe.
    • To protect against malicious client side injections.
    • To protect against malicious runtime injections.
    • To investigate file caching and prevent any malicious possibilities from the same.
    • To prevent from insecure data storage in the keyboard cache of the applications.
    • To investigate cookies and preventing any malicious deeds from the cookies.
    • To provide regular audits for data protection analysis.
    • Investigate custom created files and preventing any malicious deeds from the custom created files.
    • To prevent from buffer overflows and memory corruption cases.
    • To analyze different data streams and preventing any vulnerabilities from these.
  •  Usability testing: The usability testing process of the Mobile application is performed to have a quick and easy step application with less functionality than a slow and difficult application with many features. The main objective is to ensure that we end up having an easy-to-use, intuitive and similar to industry-accepted interfaces which are widely used.
    • To ensure that the buttons should have the required size and be suitable to big fingers.
    • To ensure that the buttons are placed in the same section of the screen to avoid confusion to the end users.
    • To ensure that the icons are natural and consistent with the application.
    • To ensure that the buttons, which have the same function should also have the same color.
    • To ensure that the validation for the tapping zoom-in and zoom-out facilities should be enabled.
    • To ensure that the keyboard input can be minimized in an appropriate manner.
    • To ensure that the application provides a method for going back or undoing an action, on touching the wrong item, within an acceptable duration.
    • To ensure that the contextual menus are not overloaded because it has to be used quickly.
    • To ensure that the text is kept simple and clear to be visible to the users.
    • To ensure that the short sentences and paragraphs are readable to the end users.
    • To ensure that the font size is big enough to be readable and not too big or too small.
    • To validate the application prompts the user whenever the user starts downloading a large amount of data which may be not conducive for the application performance.
    • To validate that the closing of the application is performed from different states and verify if it re-opens in the same state.
    • To ensure that all strings are converted into appropriate languages whenever a language translation facility is available.
    • To ensure that the application items are always synchronized according to the user actions.
    • To ensure that the end user is provided with a user manual which helps the end user to understand and operate the application who may be not familiar with the application’s proceedings
Usability testing is normally performed by manual users since only human beings can understand the sensibility and comfort ability of the other users.
  •  Compatibility testing: Compatibility testing on mobile devices is performed to ensure that since mobile devices have different size, resolution, screen, version and hardware so the application should be tested across all the devices to ensure that the application works as desired.
    The following are the most prominent areas for compatibility testing.
    • To validate that the user Interface of the application is as per the screen size of the device, no text/control is partially invisible or inaccessible.
    • To ensure that the text is readable for all users for the application.
    • To ensure that the call/alarm functionality is enabled whenever the application is running. The application is minimized or suspended on the event of a call and then whenever the call stops the application is resumed.
  •  Recoverability Testing: 
    • Crash recovery and transaction interruptions
    • Validation of the effective application recovery situation post unexpected interruption/crash scenarios.
    • Verification of how the application handles a transaction during a power failure (i.e. Battery dies or a sudden manual shutdown of the device)
    • The validation of the process where the connection is suspended, the system needs to re-establish for recovering the data directly affected by the suspended connection.
  •  Interrupt Testing: An application while functioning may face several interruptions like incoming calls or network coverage outage and recovery.
    The different types of interruptions are:
    • Incoming and Outgoing SMS and MMS
    • Incoming and Outgoing calls
    • Incoming Notifications
    • Battery Removal
    • Cable Insertion and Removal for data transfer
    • Network outage and recovery
    • Media Player on/off
    • Device Power cycle
  •  Laboratory Testing: Laboratory testing, usually carried out by network carriers, is done by simulating the complete wireless network. This test is performed to find out any glitches when a mobile application uses voice and/or data connection to perform some functions.
  •  Memory Leakage Testing:Memory leakage happens when a computer program or application is unable to manage the memory it is allocated resulting in poor performance of the application and the overall slowdown of the system. As mobile devices have significant constraints of available memory, memory leakage testing is crucial for the proper functioning of an
    application.
  •  Usability testing: It is carried out to verify if the application is achieving its
    goals and getting a favorable response from users as the usability of an
    application is its key to commercial success.
  •  Installation testing: Certain mobile applications come pre-installed on the device whereas others have to be installed from the store. Installation testing verifies that the installation process goes smoothly without the user having to face any difficulty. This testing process covers installation, update and uninstallation of an application.
  •  Certification Testing: To get compliance certificate each mobile device needs to be tested against the guidelines set by different mobile platforms.
  • Other Important Checks:
    • Uninstallation testing (whether the application can be uninstalled in a reasonable amount of time and with required criterion)
    • Check Unmapped keys
    • Check application splash screen
    • Continued keypad entry during interrupts and other times like network issues
    • Methods which deal with exiting the application
    • Charger effect while an application is running in the background
    • Low battery and high performance demand
    • Removal of battery while an application is being performed
    • Consumption of battery by application
    • Check Application side effects

Mobile Automation Overview

Mobile Automation

Mobile testing is the very unique and wide platform to learn. It consists many types of testing e.g. Functional/Network/Compatibility/Conformance/Usability/Installation/Provisioning/Battery Usage/Memory Management etc.. One need to have good understanding of Mobile OS available in the market and their behavior.
 Mobile applications are divided in 3 types:

1.  Native Application: A native app can only run on one mobile operating system such as iPhone, Android, Window Phone and BlackBerry. For using it user have to install it from market.
2.  Web App (wap): Web application can run within a mobile device’s web browser. No need to install it, this kind of application delivered on fly as needed via internet.
3.  Hybrid app: A hybrid app is built using web technologies, and then wrapped in a platform-specific shell that allows it to be installed just like a native app. Facebook is the good example of the hybrid application.
There are many challenges in mobile testing like:

  • Different range of mobile devices with different screen sizes and hardware configurations like hard keypad, virtual keypad (touch screen) and trackball etc.
  • Wide varieties of mobile devices like HTC, Samsung, Apple and Nokia.
  • Different mobile operating systems like Android, Symbian, Windows, Blackberry and IOS.
  • Different versions of operation system like iOS 5.x, iOS 6.x, BB5.x, BB6.x etc.
  • Different mobile network operators like GSM and CDMA.
  •  Frequent updates – (like android- 4.2, 4.3, 4.4, iOS-5.x, 6.x) – with each update a new testing cycle is recommended to make sure no application functionality is impacted.
 
Here is some of the areas to test in Mobility:-



Regarding Tools for Mobile Testing, here is the overview:-