How Run Effective User Acceptance Testing

Jan Nayak Singh

Test Analyst

May 21, 2018

What is User Acceptance Testing?

User Acceptance Testing (UAT) is the software testing phase during which the system is tested for acceptability and validated for the end to end business flow. UAT is conducted by the end user or the client in a separate environment, similar to the production environment. During UAT, you test end-to-end scenarios and this typically involves running a suite of tests on the completed and integrated systems.

Some Key Points about UAT

  1. This testing is to be conducted in the final stage of Software Development Life Cycle (SDLC) prior to system being delivered to a live environment.
  2. The Acceptance testing is regarded as “black box” testing, which means UAT users are not aware of internal structure of the code, they just specify the input to the system & check whether systems respond with correct result.
  3. UAT is preferably performed after System Testing is done and signed off. All Sev1 and Sev2 defects have been fixed and retested.

Fundamental Best Practices

One of the critical pain points in UAT is how late in the cycle this testing takes place. Typically UAT is one of the last efforts to ensure the quality before the product goes live. Keeping this fact into consideration, it is imperative that the UAT should be carefully planned. The following workflow can come handy and prove useful to ensure a smooth UAT cycle.

  1. Planning
  2. Execution
  3. Documentation and Reporting
  4. Evaluation


Planning

An important truth for any process where quality must be determined is the fact it should be thoughtfully planned and the same goes for UAT as well. To effectively plan the UAT phase, it  needs to be included in the master project plan and scheduled across the life of the project. The following checklist gives an overview of what needs to be planned for a successful UAT cycle.

  • Determine the point of contact for directing UAT testing.
  • Determine the  UAT team. A mix  team comprising the end users, members from the  business analysts team and QA team is a good option.
  • Gather all the requirements that will be verified as part of the UAT phase. If  an Agile process is being followed, the Acceptance Criteria can be the reference and usually provide the necessary details.
  • Setting up the Environments for UAT is another key factor in UAT testing: Where is this test going to be conducted ? Conducting the UAT in the regular development and QA environments should be avoided – these change frequently and any disruption in the development process could impact the schedule. Ideally, a separate environment for UAT would work best.It is crucial to set this up well in advance of the UAT to be sure everything is working. To iron out any inconsistencies, the QA team should do a smoke test before handing it to the customer for UAT.
  • Determine the schedule and communicate the UAT testing dates. The UAT should be planned in a manner giving some buffer for fixing any critical defects identified before the production release.

Execution

The execution phase of the UAT is the one that definitely holds the most anxiety of the UAT process. What if the customer doesn’t like it? What if there is a major issue? What if there is a requirement gap? And so on …. So if the implementation team is not prepared for the UAT and have questions about things going wrong, then cancel it!

The execution phase is an opportunity to get closer to your client and do not miss this to make client a raving fan of yours. This implies that full disclosure is important in UAT. If there is a major bug,  don’t hide it – openly discuss it and work around it. 

It is helpful to have the basic test cases written, reviewed and signed off by the responsible stakeholders. The test cases should be made available in a test management system that tracks execution. This however, does not negate the importance of allowing customers to interact with the system in an exploratory manner but rather, provides a framework establishing the guidelines within which the UAT testing is conducted to ensure all the requirements have been reviewed and tested. Always remember: Clear expectations are key for a successful UAT. The following checklist gives an overview for executing UAT in a successful manner:

  • Assure that the system is ready to be UAT’ed by the  customer. At the minimum all the Sev1 and Sev2 defects from the SIT cycle should have been closed. Any Sev3 or Sev4 defects would have to be communicated to the client in the SIT Test execution report.  
  • A technology team/person should also be arranged for supporting the UAT activities. The technology team should be responsible for clarifying any queries raised regarding functionality or any bugs reported during the UAT execution. A developer and a senior QA from the implementation to start with at least.
  • Track the test case execution and record all results and queries raised.
  • Ensure the UAT team challenges the system by executing the scenarios outside of the happy path.
  • Treat this execution as an opportunity to build relationship and trust with the client.

Documentation and Reporting

Ideally documentation and execution should happen in parallel. Recording and documenting the test results, findings, bugs and other relevant details is beneficial in evaluating the UAT execution. It is recommended to set up a system which lets you document all relevant information without losing any data. At the end of the UAT cycle a test execution report should be prepared to document all the details like:

  • What was the scope and coverage of the UAT?
  • How many testers were involved in completing the test execution complete the test cases?
  • How many defects encountered and the severity of each defect?
  • What is the current status of those of defects. Have those been fixed or deferred for next release?
  • Observations
  • Requested changes
  • Requested additions

It is worth mentioning here that the UAT test team must provide daily status updates to the stakeholders regarding the number of test cases executed, defects encountered and current status of those defects . More importantly, any Sev1 or Sev2 defects should be reported and triaged immediately for implementation team to provide the fix. This will ensure that there is minimal impact to the release schedule.

Evaluating Test Results

Once the test execution is completed and the test results documented, it is now time to evaluate and retrospect those results and processes. When evaluating the test results and findings, it gives us an opportunity to find answers to some of these questions.

  • What went well during the UAT?
  • What are processes that can be improved on?
  • What challenges we encounter and what could be the remedial actions?
  • Has any test case failed? Why the test case failed?
  • Did the test case fail because there was a requirement gap or missed during SIT phase
  • What was our  approach for change control process? Did we accept  improvements during the UAT phase that impacted the release schedule.
  • What was the approach of each tester? Did he stick to the test cases or performed exploratory testing?
  • What was the overall sentiment of the UAT team when doing the execution? Were they skeptical or felt comfortable during the execution
  • Answer to these questions will help in gathering insights and lessons learned which will help  to improve your future UAT test cycles.

Conclusion

User Acceptance Testing isn’t easy or trivial. A successful UAT cycle depends on solid planning and execution throughout the project and not just at test time. But with the right approach and planning, chances of UAT being a successful event increases significantly.

What did you think? Let us know your thoughts!
recent tweets
Need a custom solution?
Drop us a line. We'd love to help.
contact us
ProQuest Consulting is a leading Salesforce partner. ProQuest works to empower companies in Australia and New Zealand to achieve higher performance through the delivery of tailored Salesforce solutions using disciplined Agile Scrum methodology.
Get in touch.

Get social.