Testing your ERP system to perfection

It would be easy to expect that after tens of thousands of business implementations ERP software will be mature enough to just run the “install wizard” to implement an ERP system. Unfortunately, ERP systems are so flexible and complex that a huge success factor for an ERP implementation is aggressive and extensive testing.

Testing is an iterative process, which means not only testing, but evaluating problems and fixing things as you proceed. It means having valid master data available to support the test, as well as understanding those processes you are testing. It also means testing development objects and watching how they interact with everything else. Finally, it means having a dedicated test client, which can be controlled and monitored. Ultimately each test leads to a bigger test, so there is a natural progression of testing sophistication over the course of an implementation.

There are many models of testing, the most common being the ‘V’ model. This matches the design phase of the project to key test events and helps ensure full and thorough testing.

Unit testing

After initial configuration is completed, all planned transactions must be tested for simple transactional validity, so ask questions like:-

• Can we enter a sales order?
• Can we replace a purchase order?
• Can we report on production?

You will need to test, test and test again. This level is typically done on an individual basis by each functional team. All transactions, especially bespoke ones, need to be successfully tested to ensure they operate as expected.

System testing

The testing carried out to analyse the behaviour of the whole system according to the requirement specification is known as system testing. The test cases are designed based on risks and/or requirement specifications. In some cases, business processes, system behaviour, and system resources can also be taken into consideration. This is considered the final test of the software by the development team.

Integration testing

The process of combining and testing multiple components together is known as integration testing. This is a systematic approach to test the complete software structure from several unit test modules. It assures that the units are operating properly when they are combined. The aim is to discover errors in the interface between the components. The interface errors and the communication between different modules are tested during this phase.

As integration testing ends it is important to involve subject matter experts and super users. During this latter stage, real world data is introduced. The randomness of data and people ensures the right processes were identified in the integration test phase. Using realistic data allows business participants to help evaluate the output. During this round of testing, master data must be well developed, because the ability to enter real orders with real customers means that master data is available to feed the process.

User acceptance testing (UAT)

UAT directly involves the intended users of the software. UAT can be implemented by making software available for a free beta trial on the Internet or through an in-house testing team comprised of actual software users.

The following steps are typically involved in in-house UAT:

Planning: The UAT strategy is outlined during the planning stage.
Designing test cases: Test cases are designed to cover all functional scenarios in real-world usage. They are designed in a simple language and manner to make the test process easier for the testers.
Selection of testing team: The testing team is typically comprised of real end-users.
Executing test cases and documenting: The testing team executes the designed test cases. Sometimes it also executes some relevant random tests. All bugs are logged in a testing document with relevant comments.

Bug fixing: Responding to the bugs found by the testing team, the software development team makes final adjustments to the configuration/code to make the software bug-free.
Sign-off: When all bugs have been fixed, the testing team indicates acceptance of the software application. This shows that the application meets user requirements and is ready for use.
UAT is important because it helps demonstrate that required business functions are operating in a manner suited to real-world circumstances and usage.

Seek out the problem solvers in your organisation

The biggest follow-on benefit, after completing UAT, is the increase in the size of the problem-solving team at go-live. A group of key users have already seen the system, understand its complexities and have seen a resolution to any issues raised. These individuals tend to go on to become “problem solvers”; those people who can identify issues at a glance and find a fast, effective solution. These are not to be confused with “problem identifiers” though. These are the people who like to complain but are not so quick to find a solution. These are the people who can bring morale down. Seek out the solvers and the fixers, and nurture them. They will be your ambassadors after go-live.

Test entry and exit criteria

Test entry and exit criteria are required to start and end the testing phases. This is an absolute must for the success of this phase. If you do not know where to start and when to finish, then goals are not clear. By defining entry and exit criteria you define your boundaries.

Entry Criteria

The entry criteria define what is needed and necessary to start the testing phase. In general, entry criteria are a set of conditions that permit a task to be performed.

Entry criteria might include:

• All developed code must be unit tested. Unit Testing must be completed and signed off by the development team
• Functional and Business requirement should be cleared, confirmed and approved
• Test plan, cases and scripts should be reviewed and approved
• Test environment/test software is prepared
• Test data should be available
• The application should be a  to the test team
• Testers have significant knowledge of the application.
• Test resources should be available

Exit Criteria

The Exit criteria is a set of conditions based on which you can say a test event is finished. This can be difficult to determine. Many ERP solutions are very complex, and theoretically, could continue forever. When to stop testing is one of the most difficult questions to agree on.

Exit criteria might include:

• Test cases completed with certain percentage passed
• Coverage of code/functionality/requirements reaches a specified point
• All defects are fixed or closed
• Closure reports are signed off
• The risk in the project is under acceptable limit

Non-functional testing

Non-functional testing is the testing of a software application or system for its non-functional requirements: the way a system operates, rather than specific behaviours of that system. This is a contrast to functional testing, which tests against functional requirements that describe the functions of a system and its components. The names of many non-functional tests are often used interchangeably because of the overlap in scope between various non-functional requirements. For example, software performance is a broad term that includes many specific requirements like reliability and scalability.

Non-functional may include:

• Baseline tests
• Compliance tests
• Documentation tests
• Endurance tests
• Load tests
• Localisation tests
• Internationalisation tests
• Performance tests
• Recovery tests
• Resilience tests
• Security tests
• Scalability tests
• Stress tests
• Usability tests
• Volume tests

Testing Checklist

• Have you planned your testing from the outset of the ERP programme?
• Do your team understand the phases of testing the project will transition through?
• Have you explained the responsibilities during each test phase?
• Have you identified your problem solvers?
• Do you have concrete definitions of your test entry and exit criteria? Have these been communicated?
• Do you have a list of the scenarios and scripts that you wish to test? Have these been agreed/acknowledged by the business community?

This chapter was taken from our publication ‘8 steps to a successful ERP’.

Click here to download the full 32-page report on all aspects of project success.