It has been said that the law is a refuge for people who are afraid of numbers. Despite this, many practitioners (particularly those dealing with large construction disputes) are required to grapple with complex calculations and even more complex Excel spreadsheets daily. This is due to the fact that a large construction case can cover thousands of claims for payment arising from variations to the works, or claims for defective work, as well as a huge number of instructions, requests for information and contract notices (to name a few examples). Often, the sheer volume of information makes it both impractical and disproportionate for a party to plead to every event which comprises its case. Where this happens, it may be appropriate for parties to utilise statistical sampling as a means of presenting their case. Further to our post setting out our Ten Tips from the Trenches of Construction Arbitration, in this post we summarise two important English cases which discuss the relevant legal principles regarding sampling, and provide some pointers on how this can be used when preparing claims for arbitration and litigation.
Amey LG v Cumbria County Council [2016] EWHC 2856 (TCC)
Facts of the Case
This case involved a contract for highway maintenance services over seven years, during which some 36,000 individual work instructions were issued. Amey commenced proceedings against Cumbria for substantial deductions from the final monthly payment applications and Cumbria made numerous counter-claims, including in relation to defects in patching and surfacing works conducted by Amey.
With each category involving around 1,000 individual items, Amey tried to rely on extrapolation from defective samples to establish the relevant breaches and prove up their costs for remedial works. The Court accepted this on the basis that it would be impracticable for Amey to inspect and prove each defective item.
The Court’s Findings
The Court began its discussion with reference to the expert evidence on general principles of sampling and extrapolation. Two major sampling methods were described:
(a) Probability sampling: a method which identifies elements for selection through a randomised process, and the random error in the process can be accounted for in terms of confidence intervals; and
(b) Non-probability sampling: a method which relies on subjective judgments as to which elements are selected as the sample.
The Court held that in principle, non-probability representative sampling is an acceptable method of sampling, particularly given the difficulty and costliness of a carrying out genuine random sampling exercise. However, it must be demonstrated that such sampling is sufficiently representative to enable the Court to rely upon it and justify extrapolating the results to the whole population.
Guidance from the Court
In this regard, the Court accepted expert evidence from Amey as to how such a sampling exercise should be conducted to fulfil the above requirement:
(a) there should be very careful advance planning to ensure that the sample can be defended as properly representative;
(b) any issues which arise during the course of the exercise which might cast doubt on its representative nature should be carefully identified and addressed;
(c) the aim is to get as close to probability sampling as is reasonably possible, in order to remove subjective contamination and any actual or potential bias; and
(d) where it is not possible to remove bias entirely, the presence of bias should be properly recognised and addressed through appropriate steps.
Ultimately, the Court held that the extrapolated claim was not made out and identified the following major problems with Cumbria’s sampling process:
(a) the purpose of the sampling was initially intended to ascertain the presence of specified states of disrepair in patching, rather than for extrapolation;
(b) there were numerous sources of potential or actual bias in the sampling methods. For example, there were proportionately fewer samples of patching works in earlier years and in certain areas, and pre-surface dressing patches were not inspected which caused as much as 90% of the total population being screened out;
(c) no consideration was given as to the size and composition of the samples despite the substantial variations, or as to how to address and account for the effect of any bias noted above; and
(d) there was a difference in quality and lack of clarity in relation to the inspection process, and separate sampling exercises undertaken at separate times with separate processes were combined.
Unfortunately for Cumbria, the extrapolated claim was pleaded as the only basis of claim, without pleading a separate or fall-back claim for damages as to the defective samples individually, and it was not allowed to depart from its pleaded case later in the proceedings.
Standard Life Assurance Limited v Gleeds (UK) (a firm) [2020] EWHC 3419 (TCC) (“Standard Life”)
The subsequent decision of Standard Life involved a professional negligence claim against several firms that serve as the design team for the building works, which included a £25 million claim extrapolated from 122 samples taken from 3,600 variations to a building contract. The defendants sought to strike out the claim primarily on the ground that the damages were calculated on the basis of an impermissible extrapolation.
Once again, the Court accepted that it would be impracticable to scrutinize each and every variation and therefore extrapolation from representative samples was a permissible method to draw inferences and establish liability and causation of damage.
In arriving at this finding, the Court also rejected the defendants’ argument that extrapolation is limited to “systematic” breaches only. While recognising that cases of chaotic or non-existent performance may be less suited to extrapolation, the Court suggested that extrapolation may be applicable in other kinds of cases, such as where the defendant operates no system at all, but fails to perform properly in the same way repeatedly.
The Court held that there was likely to be “wheat” as well as “chaff” in the plaintiff’s claim, but a significant part of sample claim was likely to be valid and accordingly rejected the defendants’ application to strike out the extrapolated claim in its entirety.
The Court also gave various directions for case management. The Court suggested that the sample of 122 variations was inadequate and gave directions for further sampling, permitting the defendants to each select some further samples themselves to make the sample more representative of the claim as a whole.
Key Takeaways
The following lessons can be drawn from the two cases above:
- The use of sampling evidence and extrapolation is acceptable in principle and not limited to systematic breaches, but parties may have to demonstrate its necessity with reference to the impracticality of pleading each item individually. It is also a good idea to plead an alternative / fall-back claim for damages on the basis of the defective samples in case the extrapolated claim fails.
- Where possible, random probability sampling is preferable, but it is not the only permissible method. The ultimate test is simply whether the sample is sufficiently representative to justify extrapolating the results to the whole claim, whether selected through probability or non-probability sampling.
- The sampling method should be carefully designed and implemented. In particular, any potential sources of bias should be clearly identified and properly addressed in the process, and parties must be prepared to discharge the burden of demonstrating this to the Court.
- Parties should try to engage early on with a view to designing the sampling method and selecting the samples so as to reach a consensus as far as possible and minimise disputes in Court. Opposing parties should be cooperative and make constructive suggestions rather than adopt a solely confrontational position.
