Understanding PT statistics

Significance of the topic

The statistical treatment of proficiency testing (PT) results is central to interpreting interlaboratory comparisons, detecting bias, and assessing laboratory competence. Clear understanding of location, dispersion and the associated uncertainties enables laboratories to select appropriate PT schemes, interpret their performance meaningfully, and make informed decisions about method bias, repeatability and comparability across methods.

Objectives and overview of the guide

The document aims to help participants in quantitative PT schemes interpret the statistical parameters presented in PT reports. It stresses that PT reports should explicitly state how the assigned value (x_pt), its uncertainty u(x_pt) and the standard deviation for proficiency assessment (σ_pt) were determined. The guide summarises commonly reported summary statistics related to location, dispersion and the uncertainty of the location estimator, and outlines practical implications for PT participants.

Methodology and statistical approaches

The guide describes estimators for two primary features of a PT data set: location (central tendency) and dispersion (spread). It follows ISO 13528 recommendations and contrasts classical and robust approaches.

Location estimators:

• Arithmetic mean — simple and efficient for near-normally distributed data; requires outlier screening when used.
• Median (robust mean) — more resistant to outliers and asymmetric data distributions.
• Algorithm A and Hampel estimator — robust statistical methods recommended by ISO 13528 for datasets where outliers or asymmetry are possible.

Dispersion estimators:

• Classical standard deviation (s) — appropriate when data are approximately normal and outliers have been removed.
• Scaled median absolute deviation (MADe) and normalized interquartile range (nIQR) — robust measures scaled to be comparable to standard deviation under normality assumptions.
• Robust standard deviations (s*), Qn and Q methods — provide robust spread estimates with varying trade-offs between efficiency and resistance to outliers.

Standard uncertainty of the location:

• For the arithmetic mean: u(x) = s / sqrt(p), where s is the classical standard deviation and p is the number of reported results.
• For robust estimators: ISO 13528 recommends using u(x) = 1.25 * s* / sqrt(p), where s* is the robust standard deviation from MADe, nIQR, Algorithm A, Qn or Q method. The factor 1.25 is a conservative adjustment to reflect additional uncertainty for non-classical estimators.

Key results and discussion

The guide emphasises several operational conclusions that PT participants should consider:

• Robust estimators are strongly recommended when data symmetry cannot be assumed or when outliers may be present.
• If the arithmetic mean is reported, the PT provider is expected to have checked the dataset and removed outliers before calculating the mean.
• All estimators (location or dispersion) become unreliable when the number of participants (p) is small; uncertainty increases and breakdown points become more influential.
• Significant differences between the estimated location and an external reference value may indicate systematic bias associated with specific analytical methods; providers may report method-specific locations when different methods are used by participants.
• Dispersion is commonly reported in the same units as the measurand or as a percentage, and is normalised in many robust estimators to allow comparability with classical standard deviation.

The guide also summarises pros and cons of each estimator class in terms of simplicity, robustness to outliers, resistance to minority discrepant groups (RMM), efficiency for normally distributed data, and breakdown point. These trade-offs underpin the PT provider’s choice of summary statistics and influence how participants should interpret results.

Benefits and practical applications

The described statistical framework helps laboratories to:

• Select PT schemes whose statistical approach aligns with their analytical methods and expected data distributions.
• Interpret reported performance indicators correctly, distinguishing between random scatter and method bias.
• Decide whether method-specific assessment is necessary when multiple analytical techniques participate in a scheme.
• Assess the reliability of location estimates via the reported standard uncertainty and judge the significance of deviations from reference values.

Future trends and opportunities

Expected developments and opportunities to improve PT statistical practice include:

• Wider adoption of robust and hybrid statistical estimators that balance efficiency and resistance to outliers, especially for heterogeneous PT panels.
• Greater transparency and standardisation in reporting (explicit declaration of x_pt, u(x_pt), σ_pt and the algorithmic steps used), following ISO 13528 guidance.
• Use of simulation, resampling and Bayesian methods for more realistic uncertainty quantification, particularly for small p or multimodal datasets.
• Improved metadata reporting (method classes, matrix information) enabling routine method-specific analyses by PT providers.
• Automated quality checks and reproducible, open-source implementations of Algorithm A and other robust estimators to reduce ambiguity in PT reports.

Conclusion

Understanding the statistical underpinnings of PT reports is essential for meaningful interpretation of interlaboratory comparisons. Participants should review the PT provider’s chosen estimators and outlier policies, consider the reported standard uncertainty of the location when comparing against reference values, and be cautious when participant numbers are small. ISO 13528 provides the accepted baseline for method selection and uncertainty treatment, and robust approaches are increasingly recommended where data irregularities are possible.

References

1. Brookman B., Mann I. (eds.), Eurachem Guide: Selection, Use and Interpretation of Proficiency Testing (PT) Schemes, 3rd ed., 2021.
2. Eurachem leaflet, Understanding PT performance assessment.
3. ISO 13528:2022, Statistical methods for use in proficiency testing by interlaboratory comparison.