Best Practices – Building Robust Test Automation Framework

This article explains the best practices to be considered in building robust test automation framework.

Test Automation – A Brief History:

I still remember – It was sometime in 2004. I was thrilled when I first saw QTP (now it is UFT) playing the recorded script – launching a browser, entering test data and submitting the page etc. QTP was dominating the test automation world at that time and It was very expensive. Not everyone could afford. That is when I had started my career in IT.

Most of the people had no idea about test automation frameworks at that time. Developers were not involved in test automation & QA people were mostly doing record and playback with simple parameterization. They would come up with some best practices & improve their tests based on their past mistakes. Mostly the scripts required more maintenance work due to poor design. Most of the QA people would not have any idea about the design patterns/principles.

Suddenly the mobile devices started popping up everywhere, cloud technologies came up, many complex web-based applications were built, agile methodology was followed…etc. During this time everyone understood the importance of test automation. Once Selenium WebDriver has been introduced which is FREE and Open source in the same time, It got everyone’s attention. Some QA/Dev became SDET (Software Development Engineer in Test), introduced design principles and built better frameworks.

Role Of SDET:

SDET’s role is very significant in building a robust test automation framework. Once I had a chance to take a look at the test framework created by a Software Architect who was architect of the application under test. They had implemented all the latest technologies like micro services at that time which was great. But the test framework was created like a unit/integration testing framework. They were using DB dump files for data provisioning. Before they start the automated test, they need to get rid of all the data and table structure from the DB and they recreate all the test data using the file. This process used to take 30-45 mins every time.  Obviously they were able to run their tests only in the lower environment. They can not run these tests in the higher environments due to DB access. The tests were tightly coupled with this DB import process.

In order to build a robust test automation framework, We should have both the mindset of both QA + Dev & have a very good understanding of 

• OOPs concepts
• Design principles
• Design patterns
• QA specific challenges

Basic Requirements of a Test Automation Framework:

• Tests should be easy to read and follow.
  • I personally do not prefer any ‘new’ keywords in my tests. No variables / objects initialization in the tests.
  • Anyone/less technical person should be able to contribute in test automation
  • Keywords/BDD style should help here
  • Check the examples here
    • Test & Test Data Design
    • Fluent Style Page Object
• Ability to add new tests
  • Time to add a new test into the existing suite should be less.
  • Design patterns / SOLID design principles should help here.
  • Use business keywords to build new tests like login,logout, search, order, edit, cancel etc
  • Check the examples here
    • Selenium Hybrid Framework
    • UFT Hybrid Framework
    • REST API – Data Driven Approach
• Abstract away low level commands.
  • Use Advanced Page Object design pattern
  • Tests should talk to Page Objects, Page Objects talk to Page Forms / Components / Fragments which in turn talks to WebDriver API
  • There should not be any driver.findElement() in the tests.
  • No thread.sleep anywhere in the tests
  • Use fluent wait APIs for page synchronization

• This is Optional – Separate locator from page objects
  • This is for static pages with hundreds of elements
  • Separate the locators from the page objects – keep them in a separate files like properties /XML / JSON / CSV
  • Check the examples here
    • Scriptless Page Object
• Separate test data from tests
  • Do not use any hard coded test data in the tests / page objects
  • Production like test data can be generated at random
    • Check here.
  • Test data should be maintained in a separate files like XML / JSON / CSV
  • This will make your tests data-driven
• Easy data provisioning
  • Tests might expect some data to present in the application. Such data should be created by the tests itself automatically.
  • Each tests should have a setUp and tearDown methods to create and delete the test data.
  • Avoid direct DB calls for data provisioning which will make your tests run only in the lower environments.
  • Use the application APIs – This is the best approach for data provisioning.
    • Check this example
  • Use the application features like import/export for data provisioning.
• Easy maintenance
  • In case of any changes in the application, the tests should require easy maintenance.
  • Single Responsibility responsible + Page fragments/components should help here.
• Ability to integrate with other tools like junit/testng/ant/maven/jenkins etc
  • The tool/framework we choose should support CI/CD process.
• Version Control
  • Keep track of all the changes
  • I would suggest you to use Git
• Ability to run the automated test scripts in any given environment like QA, Staging, Production
  • As part of CI/CD process, the test automated tests should be able to run in any environment – including production.
  • Use of config files – one of each test environment
    • How To Run Tests In Multiple Environments
  • Jenkins should be able to override these configs at run time.

• Containerizing your tests.
  • Your tests might have a lot fo dependencies. Setting up test machines with all the dependencies is very time consuming.
  • You could dockerize your tests with all the dependencies.
  • Check this example
    • Running Automated Tests Inside Docker Container
• Ability to perform multiple types of testing
  • from a simple smoke test to full regression tests
• Ability to execute specific suites/modules
  • Huge automated tests should be categorized based on the modules / business functionalities – so that a specific module can be run at any time.
  • Each suite/module should be a separate job in Jenkins
    • Running Multiple Test Suites using Docker Compose
• Ability to execute tests in parallel
  • In order to minimize the overall test execution time, our test automation framework should support parallel test execution.
  • We need to be careful that the test data used by 1 test does not affect other test which could run in parallel.
  • If tests are dependent for some reason, then they all should be in the same suite.
• Continuous Integration / Delivery process
  • The automated tests should get triggered as part of CI/CD pipelines.
  • As soon as the new version code is promoted to an environment, a smoke test should also be performed automatically.
  • Based on the changes in module of the application, a corresponding regression suite should get triggered.
  • It helps us to reduce the overall QA cycle time drastically.
•  Easy to share the results
  • As soon as a smoke test is performed, send out an email notification with the status to all the team members
  • Real Time Reporting approach
    • Real Time Test Results Using ElasticSearch & Kibana
  • Create a nice HTML report with steps performed to send out an email.

• Easy to debug
  • Report should have all the steps performed with time stamps + screenshots for easy debugging the failed tests.
  • Results history should be maintained in a separate DB

 

Summary:

We should consider the above capabilities while designing our test automation framework to come up with a more reliable, robust automation scripts to test products/features we launch. By using business keywords/feature files, we should be able to add new tests. CI process helps us to detect any defects as soon as it has been introduced. Thus overall time to market is greatly reduced.

 

Happy Testing & Subscribe 🙂