Test Script Notes
User Acceptance Testing
Under construction
User Acceptance Testing should be used to test that a product is satisfactory to an end-user or customer, though its role and form do vary from company to company. The user should be involved in the testing which should be designed to enable him or her to determine if the system meets their business requirements and will work in their business environment. The user should gain confidence in the system. If the system turns out not to be what the user needs, then you have made a very expensive mistake. It is therefore important to focus on user acceptance testing right from the start. The system should not only achieve its stated purpose, but also fit well into the business, not making processes harder or slower or inadequate, nor requiring unnecessarily highly skilled (and expensive!) staff. The user is the best person to judge this. The system should also be stable.
Often, User Acceptance Testing instead takes one of the following forms, which do not meet the objectives:
- A repeat of a previous phase of testing (which had different objectives) involving the user
- A quick demonstration to the user, with little opportunity to find problems
- A pre-release formality
The following are all important for User Acceptance Testing:
- start thinking about testing right from the start of the project.
- agree acceptance criteria in advance with users and managers.
- the users and managers should buy into the purpose of the test. The earlier they are involved in test planning, the better. Expectations should not be too high or low.
- the users should be familiar with the business processes.
- avail yourself of the user's knowledge of the business and data.
- testing should be against the business requirements - all of them.
- tests should be described in language all involved can understand, not in jargon.
- arrange for any training of participants that may be required.
- the tests should be designed to meet all the objectives, and only the objectives. Keep the number of tests to the minimum.
- Black box testing - no need for users to know how the programs work
- the tests should not be redundant repeats of earlier tests.
- the tests should not take more time than the user can spare, but should be sufficient for the purpose.
- All testing should be carried out using the applications, not by hacking data.
- determine how many people will be required, with what knowledge, to finish the task in time.
- tests should be scheduled earlier enough that they can fit into the time the user can spare, and be completed in time.
- If they don't know already, users may need to be trained in the purpose and form of UAT, what their responsibilities are, a method of showing the business processes graphically and breaking them down for testing, and in the use of the system under test.
- determine what standards to use. Train people in them if necessary.
- If more than one group testing, determine ways of making sure your tests do not affect each other.
- the system should be in good enough a state that the user's time will not be wasted nor will they suffer more frustration than can be helped (and to spare your blushes!).
- determine what tools to use for testing or test tracking. Acquire them and train people in their use, as necessary.
- a system should be in place to log issues, communicate them to all who need to know, resolve them promptly, and communicate the outcomes. Someone should be designated responsible for this.
- the system should be one that the user will be happy with.
- problems should be prioritised, so that major revision is not done for minor problems. Making changes at this late stage can be very expensive.
- testing should include online help, batch procedures (e.g. end of day)
- a clear chain of upper management should be designated to make policy decisions - testers should not do this.
- tests are repeatable, so problems can be reproduced.
- tests should simulate real-life situations.
- Check that training, online help, user guides etc. are (kept) in line with the current version of the system.
- testing should be well documented. Bear future regression testing in mind, as well as audit.
- test plans should include data to be used, and expected results.
- show consideration to developers, who have worked hard to produce the system. Avoid "Us" vs. "Them".
- on successful completion, the tests should be signed off.
- requests for future enhancements should be flagged for later action, not just mixed in with fixes needed before the current phase of the system goes live.
- review the going-live procedures, training materials, etc. in light of UAT experience.
- if there is a QA department, they should be copied all documentation for review
The techniques for planning user acceptance tests is the same whether the product has been developed in house or by a vendor.
Steps:
- Define test strategy, agree deliverables
- Create requirements, objectives, acceptable margins. Prioritise.
- Review requirements, objectives, acceptable margins
- Create outline test plan
- Review outline test plan
- Create test specifications
- Review test specifications
- Create test cases
- Review test cases
- Create test matrix
- Review test matrix
- Select tools
- Estimate and allocate time (enough for discussion too), resources, cost
- Set up environment
- Load data
- Check you are testing with right environment, system, data
- Add users and reference data using the methods to be used in production.
- Run tests (manual, automated)
- Track issues
- Document test results
- Analyse test results. Metrics can help predict future test effort and corrective maintenance.
- Deal with change requests (go back to relevant earlier step if changes made)
- Decide whether to release the system. User should sign off system. Know when to stop!
- Write final report
- Train remaining users.
- Consider Pilot release or phased release
- Build live system in live environment, ensure it is as signed off, test that it still works. If UAT was in test environment, check that all test data is removed, and all live data is loaded. There should be provision for receiving and dealing with feedback.
- Post project review
Models for testing:
- Waterfall
- V-model
- RAD software development life cycle