When is outlier handling required?

It is important that the approach to handling outliers is considered at the outset of any work where this may become an issue. A clear communication strategy should be developed for every output, which will include an assessment of what, if any, outlier handling is required. Even where the intended publication will not directly identify outlier institutions it may be necessary to consider this if the underlying data would identify outliers and would reasonably be expected to be released under a freedom of information request.

The approach to outlier handling should remain proportionate and it is expected that most outputs will continue to be non-controversial and that outlier handling will be the exception rather than the rule.

The decision on the level of effort to devote to outlier handling should be informed by an assessment of the likely sensitivity of the outputs. It is suggested that the full approach outlined in DH guidance and this document should only be considered for analyses where there is:

  1. A high degree of clinical impact (e.g. mortality or outcomes with a long term impact on quality of life) and
  2. Direct influence over the process or outcome at the organisational level considered in the analysis (e.g. measurement of post-operative mortality or emergency readmissions by multidisciplinary team).

Both of these factors will always be a matter for clinical and professional judgement, but a framework for decision making is suggested in figure 1.

NCIN framework for decision making on outlier handling.

Figure 1. Suggested framework for decision making on outlier handling.

Where it is not clear whether outlier handling is required (the amber boxes in figure 1) the work should be flagged with the NCIN’s Outlier Advisory Group by emailing outliers at ncin dot org.uk link outliers@ncin.org.uk with a description of the planned analysis and a recommendation for whether outlier handling should be applied. The group will consider whether outlier handling is required and respond to confirm this. Previous decisions on outlier handling made by this group will be logged on the NCIN website to guide others. This approach will help ensure consistency in decision making until a body of precedent is established.