Successfully completing IT projects: what matters in testing and acceptance.
The concept of acceptance is a red rag for most contractors, because acceptance implies a contract for work and thus more extensive obligations than the contractor would like to undertake. However, a contract does not become a service contract by simply sweeping the unloved acceptance under the rug. Moreover, every project needs a process to determine whether the software works or not before it goes live. It therefore helps neither the legal classification of the contract nor the success of the project to simply ignore the regulations on acceptance.
Ultimately, in practice, no one can avoid acceptance. It is much better to establish a reasonable acceptance procedure than to remain silent on this aspect in the contract. In addition, acceptance is also useful for the contractor: he wants to know whether his performance is accepted in principle. His risk of damages can be considerably increased by operational use without formal acceptance. Finally, acceptance also serves to detect and eliminate errors. What to do when IT projects fail and what options are available in the worst case is described in my article IT Acceptance.
Acceptance takes time
The contractual partners are free to determine what is to be tested and accepted and how the tests are to be performed. There are no specific requirements for this.
In larger projects, acceptance tests cannot be performed in a single day; the services are so complex and the potential impact on business operations so multi-layered that a longer period of time is required to test the system. The effort for testing (before and during acceptance) usually exceeds the effort for development and implementation considerably. Many customers in particular are not aware of this.
Agreement of precise criteria
Subject of an acceptance can be all services provided by the contractor. This includes without limitation
• documentation and
• data base.
It usually makes sense to regulate the following areas of an acceptance:
• preparatory measures of the provider before provision for acceptance by the provider,
• provision for acceptance by the provider,
• acceptance test according to the agreed procedure,
• defect categories,
• acceptance protocol,
• repetitions of the acceptance and possible consequences.
Acceptance shall be based on acceptance specifications agreed between the contractor and the customer. These specifications define the requirements for the services and functions of the system due. As a rule, and ideally, the acceptance specifications should be based on requirement specifications and detailed specifications and should further detail the requirements contained therein.
The legal basis requires a performance description that is as detailed and unambiguous as possible. It is not a question of quantity, but of a careful and as complete as possible description of what the outsourcing service provider has to provide. You can find more on the subject of performance specifications in my article on IT outsourcing.
Based on the acceptance specifications, test procedures should be drafted by the contracting parties that describe how the individual functionalities of the system will be tested. Such software testing is an analytical, dynamic measure for software quality assurance.
Minimizing risks by testing
Whether a software works correctly in all functions cannot be proven by testing (except in trivial cases). Programs are usually far too complex for this. A test will never be able to cover any possible combination of all variants. In other words, (acceptance) tests can minimize the risk of serious errors in productive use, but not completely eliminating such risk.
If tests require the cooperation and provision of the customer, the contractual provisions should include an obligation to do so. Even large customers usually have scarce personnel resources and must be able to schedule and allocate personnel.
Create a test plan
A test concept or test guideline describes the specified minimum requirements for the test procedure, responsibilities, accountabilities and dependencies for the specific IT project. Furthermore, the test concept defines the framework conditions (e.g. test environment), objectives of the test procedure, the test methods and test procedures to be used, and the scope of the tests to be performed.
In the case of larger projects, the contractual partners should draw up a test plan that also specifies the resources required (such as rooms, servers, workstations, employees), the tests to be performed and the employees involved in the project, the time period and deadlines. The contractual partners must therefore reach agreements on the following topics:
• Is a test manager required; if so, who is suitable for this?
• Who shall carry out the tests?
• What is the (agreed) acceptance specification? Who will create it and when?
• Test plan (start and end).
• Determination of the objects to be tested, test cases.
• Definition of the "acceptance procedure", i.e. the defect classes or defect categories.
• Who provides the test environment? When and with which equipment?
• What tools are needed for the tests? Should a defect recording system be used and if so, which one?
Define defect categories
Some disputes can be prevented in practice by contractually defining defect categories/defect classes and their impact on acceptability. These include categories such as "preventing acceptance testing", "preventing acceptance", "impairing acceptance", "without effect on acceptance" and "to be remedied within the scope of liability for material defects and defects of title". It makes sense to provide project-specific examples in addition to a general definition of each category/class in order to facilitate the classification of the defects that occur.
When is an acceptance successful?
Contractual regulations focus almost exclusively on the maximum number of defects. Theoretically, compliance with the contract is given when all requirements of the customer have been implemented. However, checking whether this is actually the case is usually made more difficult by the fact that the requirements do not have the necessary level of detail. In this case acceptance subject to the elimination of certain defects may be most likely the more economical way than refusing acceptance because the number of defects in a certain category has been exceeded. The situation is different if the customer is unhappy with the project status anyway and is looking for reasons to refuse acceptance. As a supplier, you usually have bad cards at this point. The issue arising is whether other or at least additional criteria can be defined as prerequisites for acceptance. For example, in addition to the number of defects also the number of successfully completed test cases could be taken into account.
Use freedom of scope
The extent to which the acceptance regulation is drafted usually depends on the complexity of the project and - unfortunately - also on the time available to the contractual partners to devote to this topic. Both contractual partners should focus to good preparation and ensure that sufficient time and resources are available for acceptance. This is the only way to successfully complete a project. We advise customers and IT service providers on project preparation and help them identifying and covering the relevant acceptance issues. In addition, we support you with specific questions regarding the implementation of the project.
Michaela Witzel, LL.M. (Fordham University School of Law), Certified Expert for IT Law