Analytical Performance Specifications Based on Biological Variability

Biological variability is the variability in a parameter due to physiological differences within individuals (CV_i) and between individuals (CV_g). Each individual has random fluctuations around a homeostatic set point, but these homeostatic set points will vary between individuals.

Similarly, analytical goals may relate to monitoring a single patient or making a diagnosis based on the likelihood of belonging to a healthy or diseased group of patients. Logically, the more difficult task is to perceive a change within an individual compared to being able to tell the difference between individuals. The analytical goal for monitoring a patient is that noise added by analytical uncertainty (CV_a) should be less than half the daily biological variability of the patient (CV_a < ½ CV_i). The performance goals for diagnosis are wider and typically expressed as

Total Error (TE) = 0.25 (CV_i² + CV_g²)^½ + 1.65 x ½ CV_i.

Callum Fraser also described a fine-tuning of imprecision and total error goals from minimal, desirable and optimal. External Quality Assessment (EQA) experts have also considered that when stating total error goals for EQA programs, we should be 99% sure when we say that a laboratory has exceeded the performance goals (rather than 95% sure) so 2.33 is used as the multiple for imprecision rather than 1.65.

Therefore, the new APS are defined using biological variability principles but only adopted when the majority of participants can achieve the goals. As a general rule, a goal is adopted if over 80% of laboratories can achieve the performance as we also seek to encourage further refinement of methods particularly to achieve the tighter monitoring goals.

Analytical Performance Specification Review Process

The aim is for the APSs for all measurands to be reviewed based on the principles outlined above. Revised APS have been introduced for most of the Chemical Pathology, Haematology and Immunology programs. The review is ongoing and conducted by the specialist parties, the RCPAQAP and other experts when required.

Click on the links below to access Analytical Performance Specifications for RCPAQAP disciplines

Chemical Pathology Analytical Performance Specifications

Haematology Analytical Performance Specifications

Immunology Analytical Performance Specifications

Transfusion Analytical Performance Specifications

Analysing results with less than (<) or greater than (>) values

Participants should test proficiency items in the same manner as patient specimens and this include submission of results and the use of < and > calculations. Previously, these results were excluded from the statistical analysis for some programs and not assessed or displayed on histograms, Youden or Linearity plots. Likewise, where programs had two samples per survey, the result of the accompanying sample would not plot on the Youden.

As a result of feedback from our customers, values above or below the nominated measuring range will be included in the statistical analysis and displayed on the reports. RCPAQAP software will remove the < / >, and the resulting numerical values used to obtain the target median. The survey results will still show as submitted (e.g. <4.0 mmol/l) on the report, but will not be assessed against the related APS.

We will continue to assess if this change adds value to survey reports and welcome your feedback.

Outlier detection process when calculating Mean, Standard Deviation (SD) and coefficient of variation (CV)

The mean, SD and CV are illustrated in survey reports when analysing quantitative results. To calculate these values, a process to eliminate outliers is performed on the assessment categories before the final calculations are performed to determine the mean, SD and CV.

The process to detect outliers uses traditional statistics, where all results outside three standard deviations of the mean value are removed. Once removed, the values are recalculated to determine a revised mean, SD and CV. This process is repeated again before the final mean, SD and CV are calculated. These are illustrated in the reports.

This process is only performed on method category groups with six or more results in the dataset. Once this process is complete and outlier results have been eliminated from statistical calculations, the remainder of the results submitted will be used to calculate the final “All Method” mean, SD and CV.

Significant Figures and Rounding

RCPAQAP reports may contain values with too many significant figures for meaningful result analysis and are condensed by ‘rounding’. This is a simple procedure that reduces the number of significant figures according to established standards.

The number of significant figures considered sufficient will depend on the type of measurand. However, during calculations to perform statistical analysis, no rounding is performed until the final value is calculated.

In a simple example containing five significant figures, 1.1451 becomes 1.15 where three significant figures is considered sufficient. Note that 1.14 would be incorrect rounding, since 1.15 is closer than 1.14.

Quantitative survey results – result flagging

Survey results outside the Analytical Performance Specifications (APS) are highlighted as red or amber in survey reports and listed as high or low in the “Summary of Performance” and “Overall Performance” sections.  The colour assists in prioritising result review and are defined below.

Red = Result is outside the APS range of the ‘all result’ target.

Amber = Result is outside the APS range of the ‘all result’ target but within the APS range of the ‘peer group’ target OR Result is within the APS range of the ‘all result’ target but outside the APS range of the ‘peer group’ target.

Supervisor reports adopt this logic and will highlight results according.