English 
Spanish 
French 
Malay Histograms
Provide a frequency distribution of the results submitted by all participants. The blue bar represents the median or target result, the blue circle indicates the participant’s result.
Introduction
Analysis of Participant results
The Immunology discipline supplies a variety of report styes that provide a direct comparison of individual participant results with all results received and graphical representations showing results from the same method, instrument and reagent groups (if applicable). Participant results will be compared to a calculated median or target from all method classifications for a particular test as well as comparing results within the main variable of the method system for the measurand. Survey reports are issued after the scheduled closing date.
Analysis of results using robust statistics requires a minimum of six (6) or more results and assigned targets are determined by 80% consensus.
For all other programs, targets are qualitative (e.g. Normal, Equivocal, Elevated or Not Detected / Detected / Equivocal/ Indeterminate) or descriptive (e.g. ANA pattern). These targets are set and based on 80% consensus or in consultation with the clinical advisory committee using the clinical information available/gold standard test results. These programs provide reports that have been tailored specifically for each module.
All Immunology reports are structured to provide:
- Performance Summary and overall performance
- Result review
- Method Comparison (if applicable)
- Commentary
- Cumulative summary of the year’s performance.
Immunology Quantitative Survey reports
The Immunology discipline provides programs that are fully quantitative and will adopt the Quantitative reporting system. Examples of programs include Specific Proteins, Rheumatoid Factor, Immunoglobulin E, Phenotyping programs.
Programs that assess qualitative measurand
The remainder of the Immunology program targets are qualitative (e.g. Normal, Equivocal, Elevated or Not Detected / Detected / Equivocal / Indeterminate or descriptive (e.g. ANA pattern). These targets are set based on 80% consensus or in consultation with the clinical advisory committee using the clinical information available/gold standard test. These programs provide reports that have been tailored specifically for each module.
Quantitative assessments
Survey Reports
Survey Reports summarise every pair of specimens for each measurand and provide summary data on your performance throughout the cycle. Reports provide a graphical comparison of individual results with all results received and with participants using the same method categories – examples are the analytical principle, measurement system and reagent source.
Quantitative results are usually compared with the specific target value (if known), the “overall median” or the median of the main variable of the category median. Analytical Performance Specifications (APS) are set either side of the expected value. Non-numerical results (descriptive results) are compared with a target value or overall method group consensus.
The Analytical Performance Specifications (APS) are unique for each measurand, and the acceptable range for each specimen is calculated from the central value (target, median or weighed-in value). These ranges are displayed in the report histograms and Youden Plots. The comment “Low” or “High” is added if the result is outside the APS and will be highlighted for review.
Note: The z-score is provided as an additional parameter to demonstrate the number of standard deviations a participant’s result is away from the mean value. It measures performance based on what is achievable from test methods. Participants are not assessed against the z-score.
Report Interpretation
The new report format follows a standard structure for all disciplines that survey quantitative measurands with two samples in each survey challenge.
The survey reports incorporate linear regression analysis to determine the precision and accuracy of the testing procedure.
Some programs allow participants to submit results in SI or mass units. The report will default to the units submitted by the laboratory.
If survey results are not received by the close date an appropriate message is returned.
The structure of the report is as follows:
1. Summary of Performance
- Measurand: All measurands that are performed by the participant are presented in the summary table.
- Expected Result: The expected result lists the target result for the measurand. The target result is based on the type of target source used – these are listed below.
- Calculated ‘All method median’
- Calculated ‘Category median’, using the main variable of the test method. This could be the analytical principle, measurement system, reagent or calibrator. The target source is highlighted on the result page of the measurand.
- Specific target – A sample with a known quantity of the measurand. A specific target is usually assigned to samples that have been tested using a reference method or have been tested by a reference laboratory.
- Your result: The result submitted by the participant.
- Review: Results flagged for review are outside the Analytical Performance Specifications (APS). These are highlighted in red and are listed as ‘High’ or ‘Low’.
- Z-score /APS score: The z-score and APS scores are two types of performance indicators and calculated against the target source of the measurand, whether it is the calculated median or mean of the method being used, the calculated median or mean of All results or a Specific target.
The Z-score uses the ‘mean’ result and SD to indicate the number of standard deviations a participant is away from the average result. The APS score uses the ‘median’ and the assigned APS to calculate the APS score.
Performance assessment is based on the APS score. If greater than 1.0 the measurand will be highlighted for review in the “Overall performance” section of the report. The z-score is provided as an additional parameter to demonstrate the number of standard deviations a participant’s result is away from the mean value. It measures performance based on what is achievable from test methods. - MPS (Measurand Performance score): The Measurand Performance Score (MPS) is calculated by performing linear regression analysis on a set of samples (the minimum number of samples to perform linear regression is 6) and uses the SD (precision) and Bias (accuracy) obtained to determine the total error. This is then compared to the sample range’s median APS (MPS = 2SD + Bias / Allowable limit of Assessment). If the score is greater than 1.0, it is highlighted in bold to consider for further review.
2. Overall Performance
The Summary of Performance provides participants with an assessment of their overall performance and indicates what measurands require further review.
APS score calculation
Measurand APS: APS +/- 0.5 up to 5.00, +/- 10.0% > 5.00 x109 cells/L (5.0 is the measurand decision point)
| Lab result above decision point Lab result = 19.30 (above decision point – 10% range) Target result: 18.40 Measurand limit: 10% of 18.40 = 1.840 APS Score = (19.30 – 18.40) / 1.840 APS Score = -0.49 |
Lab result below decision point Lab result = 2.90 (below decision point – 0.5 range) Target result: 2.60 Measurand limit: 0.5 APS Score = (2.9 – 2.6) / 0.5 APS Score = 0.6 |
3. Result Review
A
B
Lab results
Lists the results and method categories returned by the participant.
C
Analytical Performance Specification (APS)
The APS that has been assigned to the measurand.
D
Youden charts
Represent a scatter of two sample results plotted against each other. The sample with the higher measurand level is on the y-axis plotted against the lower level on the x-axis. Five Youden plots are presented and illustrate from left to right, results from all results, the participant’s method, analytical principle, measurement system and the reagent. The participant’s result is highlighted by the blue dot.
Examples of Youden charts
| Biased results | Biased Laboratory |
 |
 |
E
Levey Jennings type plot (z-score)
Displays the participant’s z-score for up to 6 sets of returned results and provides an indication of the precision and accuracy of results within a survey year.
F
Levey Jennings type plot (APS score)
Displays the APS score for up to 6 sets of returned results and provides an indication of the precision and accuracy of results within a survey year.
Examples of Plots
| Imprecision | Bias |
 |
 |
G
Linearity plot
Displays the participant result against the expected result, and indicates linearity across different survey sample levels.
Examples of Plots
| Imprecision | Bias |
 |
 |
Measurand performance: The Measurand performance page provides participants with a breakdown of the results returned.
A
Histograms
Provide a frequency distribution of the results submitted by all participants. The blue bar represents the median or target result, the blue circle indicates the participant’s result.
B
Lab results
Lists the results and method categories returned by the participant.
C
Analytical Performance Specification (APS)
The APS that has been assigned to the measurand.
D
Youden charts
Represent a scatter of two sample results plotted against each other. The sample with the higher measurand level is on the y-axis plotted against the lower level on the x-axis. Five Youden plots are presented and illustrate from left to right, results from all results, the participant’s method, analytical principle, measurement system and the reagent. The participant’s result is highlighted by the blue dot.
Examples of Youden charts
| Biased results | Biased Laboratory |
|
|
E
Levey Jennings type plot (z-score)
Displays the participant’s z-score for up to 6 sets of returned results and provides an indication of the precision and accuracy of results within a survey year.
F
Levey Jennings type plot (APS score)
Displays the APS score for up to 6 sets of returned results and provides an indication of the precision and accuracy of results within a survey year.
Examples of Plots
| Imprecision | Bias |
|
|
G
Linearity plot
Displays the participant result against the expected result, and indicates linearity across different survey sample levels.
Examples of Plots
| Imprecision | Bias |
|
|
Precision and Accuracy
High / Low or Low / High = Imprecision
High / High or Low / Low = Bias
4. Linear Regression
A
Summary of results
Displays the samples used to calculate linear regression analysis, the main method category value for this program (the CD3+ Lymphocytes example displaying the “reagent-lysing method – Beckman coulter IMMUNOPREP). The table columns represent the Sample ID, Measurement System category assigned for result comparison, median result for the method category, median of ALL results, participant’s result, number of results from the method category, number of results from all results submitted, High/Low review flag (W= Within APS, no review required), Z-score and the APS score.
B
Your Linearity – compared to the target source
Linear regression analysis is based against the target source of each sample, which could be the “median of all results”, “median of the assessment category” (measurement system) or the median of a “specified target”.
When performing linear regression, the following rules of assessment are applied:
- A minimum of 6 samples are required for linear regression analysis (applies to all target sources)
- If the target source is “median of the assessment category”, there must be a minimum of 6 results in your peer group, for all survey samples, otherwise, linear regression is based against the “median of all results”.
- If there is a change of any method category, linear regression is based on the sample range from when the change occurred (applies to all target sources).
C
Your slope and intercept
Provides the slope and intercept calculated from the linear regression analysis. Also provides the target low and high level, with your corresponding low and high levels determined from the “line of best fit”.
Using the slope and intercept the values of your line of best fit are determined compared to the lowest and highest target values for the cycle.
D
Your Linearity – compared to the All and subgroup
To provide a comparison of all methods, linear regression is performed against the same data source (all results or specific target). The slopes represent the bias obtained from all laboratories, illustrating the slopes from all labs in ‘grey’, the slope from participants using the same assessment category in ‘navy blue’ and highlighting your slope in ‘light blue’.
E
Your Imprecision compared to All & Subgroup
This histogram represents the imprecision (CV%) obtained from all laboratories. It is calculated from the scatter around the target source linear regression line of best fit. The histogram illustrates all labs in ‘grey’, participants using the same assessment category in ‘navy blue’ and highlighting your CV% as a ‘light blue’ dot.
F
Your Precision and Accuracy
Provides participants with the SD, CV and Bias calculated from linear regression and how your results ranked against all laboratories that participated. Ranking (from 0 or best to 100, worst) is illustrated on the top row of the scale provided under the bar.
Standard Deviation: The SD is the standard error of the estimate (Sy.x) and can be regarded as the average SD across the range of concentrations analysed. SD provides a value in the units of the test. SD will tend to be high if you report high results and low if you report low results.
Coefficient of Variation: The SD divided by the mid-point of your laboratory’s range of concentrations, expressed as a percentage:
| CV%= | SD | x100 |
| (low value + high value)/2 |
| Average Bias: Your biases at the low value, high value and mid value are determined. These are the differences between the line of expectation (45° line) and your line of best fit. |  |
The average bias is calculated as:
| Bias = | [low bias] + [mid bias] + [high bias] |
| 3 |
G
Your Stats and Ranking
Provides participants with a table summarising their SD, CV, Bias, MPS and performance ranking.
H
Additional notes
Provides participants with information, such as the target source used for linear regression (example displaying “calculated median of all results) and lists any method category changes made across the survey sample range being analysed.
A
Summary of results
Displays the samples used to calculate linear regression analysis, the main method category value for this program (the CD3+ Lymphocytes example displaying the “reagent-lysing method – Beckman coulter IMMUNOPREP). The table columns represent the Sample ID, Measurement System category assigned for result comparison, median result for the method category, median of ALL results, participant’s result, number of results from the method category, number of results from all results submitted, High/Low review flag (W= Within APS, no review required), Z-score and the APS score.
B
Your Linearity – compared to the target source
Linear regression analysis is based against the target source of each sample, which could be the “median of all results”, “median of the assessment category” (measurement system) or the median of a “specified target”.
When performing linear regression, the following rules of assessment are applied:
- A minimum of 6 samples are required for linear regression analysis (applies to all target sources)
- If the target source is “median of the assessment category”, there must be a minimum of 6 results in your peer group, for all survey samples, otherwise, linear regression is based against the “median of all results”.
- If there is a change of any method category, linear regression is based on the sample range from when the change occurred (applies to all target sources).
C
Your slope and intercept
Provides the slope and intercept calculated from the linear regression analysis. Also provides the target low and high level, with your corresponding low and high levels determined from the “line of best fit”.
Using the slope and intercept the values of your line of best fit are determined compared to the lowest and highest target values for the cycle.
D
Your Linearity – compared to the All and subgroup
To provide a comparison of all methods, linear regression is performed against the same data source (all results or specific target). The slopes represent the bias obtained from all laboratories, illustrating the slopes from all labs in ‘grey’, the slope from participants using the same assessment category in ‘navy blue’ and highlighting your slope in ‘light blue’.
E
Your Imprecision compared to All & Subgroup
This histogram represents the imprecision (CV%) obtained from all laboratories. It is calculated from the scatter around the target source linear regression line of best fit. The histogram illustrates all labs in ‘grey’, participants using the same assessment category in ‘navy blue’ and highlighting your CV% as a ‘light blue’ dot.
F
Your Precision and Accuracy
Provides participants with the SD, CV and Bias calculated from linear regression and how your results ranked against all laboratories that participated. Ranking (from 0 or best to 100, worst) is illustrated on the top row of the scale provided under the bar.
Standard Deviation: The SD is the standard error of the estimate (Sy.x) and can be regarded as the average SD across the range of concentrations analysed. SD provides a value in the units of the test. SD will tend to be high if you report high results and low if you report low results.
Coefficient of Variation: The SD divided by the mid-point of your laboratory’s range of concentrations, expressed as a percentage:
| CV%= | SD | x100 |
| (low value + high value)/2 |
| Average Bias: Your biases at the low value, high value and mid value are determined. These are the differences between the line of expectation (45° line) and your line of best fit. | |
The average bias is calculated as:
| Bias = | [low bias] + [mid bias] + [high bias] |
| 3 |
G
Your Stats and Ranking
Provides participants with a table summarising their SD, CV, Bias, MPS and performance ranking.
H
Additional notes
Provides participants with information, such as the target source used for linear regression (example displaying “calculated median of all results) and lists any method category changes made across the survey sample range being analysed.
Assessment Criteria
The assessment criteria are defined as measurand performance. As analytical error is due to both imprecision and bias, program organisers have defined Total Error as follows:
Total Error = 2SD + Bias
The quality of your laboratory’s performance is then determined by comparing the Total Error to the Analytical Performance Specification at the mid-point of the range of measurand concentrations for the cycle as follows:
| Measurand Performance Score (MPS) = | 2SD + Bias |
| Analytical Performance Specification |
These examples of bicarbonate analyses may assist in understanding this method of assessment.
- QA specimens are as follows:
Low Level 15.0 mmol/L
High Level 35.0 mmol/L
The mid-point concentration is therefore 25.0 mmol/L. - The Analytical Performance Specification for bicarbonate is:
± 2.0 mmol/L up to 20.0 mmol/L
± 10% when greater than 20.0 mmol/L
The Analytical Performance Specification at the mid-point (25.0 mmol/L) is therefore 2.5 mmol/L.
Example – Laboratory 1
| SD = 0.8 mmol/L | Bias = 0.5 mmol/L |
| Total Error = (2 × 0.8) + 0.5 = 2.1 mmol/L | |
| Measurand Performance = | 2.1 | = 0.84 |
| 2.5 |
Note: When the Total Error is less than the Analytical Performance Specification then the Measurand Performance will be less than 1.0. This is the desired level of performance.
Example – Laboratory 2
| SD = 1.5 mmol/L | Bias = 0.1 mmol/L |
| Total Error = (2 × 1.5) + 0.1 = 3.1 mmol/L | |
| Measurand Performance = | 3.1 | = 1.24 |
| 2.5 |
An undesirable result – due predominantly to imprecision.
Example – Laboratory 3
| SD = 0.5 mmol/L | Bias = 2.0 mmol/L |
| Total Error = (2 × 0.5) + 2.0 = 3.0 mmol/L | |
| Measurand Performance = | 3.0 | = 1.20 |
| 2.5 |
An undesirable result – due predominantly to bias.
5. Method Comparison
The method comparison provides a breakdown of all the methods (assessment category) used by participants and lists the statistics calculated for the latest survey results obtained (left) as well as the Precision and Accuracy results calculated from the linear regression analysis on the sample range used, providing the median values for each method listed, facilitating peer group comparison. The full set of statistics is seen only when there are four or more values in the dataset – three values in the dataset will illustrate the median value only and method categories with two or less users do not present any statistical data.
The “Survey Report Interpretation” flowchart can be found on the RCPAQAP website under “Resources” or can be accessed by clicking this link.
Adrenal Antibodies
The following examples of reports adopt the standardised structure.
Report Format – Adrenal Antibodies
1. Summary of Performance
Method system used by your laboratory:
- Category: The method category specific to the measurand
- Method: The Method supplied by the participant associated to the method category.
Performance Assessment:
- Test: All measurands that are performed by the participant
- Your result: The result submitted by the participant
- Target: The target result for the measurand
- Assessment: Results flagged for review are not in consensus. Qualitative measurands are highlighted in red if Discordant.
2. Result Review
The result review page displays histograms of the distribution of all survey results (grey), highlighting your method (dark blue) and your result (●).
3. Method Comparison
The Method comparison section provides a breakdown of the results returned for each method used in the survey.
4. Cumulative Assessment
Provides a snapshot of participant performance over a survey year, displaying the results returned, the expected result and the Assessment provided.
Anti-Saccharomyces Cerevisiae Antibodies, Fungal and Avian Antibodies, Bee Venom IgE, Inhalant Allergens, Serum Free Light Chains, Thyroid Antibodies
Autoimmune reports are designed around common elements, although the reports differ subtly from each other.
1. Performance Summary
- Test: All measurands that are performed by the participant
- Your Result: The result submitted by the participant
- Expected Result: The expected result for the measurand (if applicable)
- Assessment: Results flagged for review are not in consensus. Qualitative measurands are highlighted in red if Discordant.
2. Result Review
A
Survey Sample and Measurand Name
The survey sample ID and the full name of the measurand.
B
Target
The target or expected result for the survey sample.
C
Survey Results & Method Classification
The survey results submitted for the qualitative and associated quantitative measurand. The method classification the laboratory has submitted. Ensure that your method classification is correct. If the method classification information provided by us does not allow for an adequate definition of your method then contact the RCPAQAP.
D
Histogram
Histogram showing the distribution of all results. Results highlighted in dark blue are the peer group results representing the main variable in the test method. In this case, the reagent. The light blue dot (●) represents the participant’s result.
E
Scatter plot
Scatterplot showing the distribution of all results (grey dots). Results highlighted in dark blue are peer-group results representing the main variable in the test method. The light blue dot (●) represents the participant’s result. The median is represented by a dashed line (—–) within the scatter of results.
F
Method Comparison
Each report will provide a method comparison table to review the performance across all methods for the quantitative and the associated qualitative measurand. Groups with less than 4 users will not provide quantitative results with an SD or CV. Groups with less than 3 users will not provide a median or mean result.
A
Survey Sample and Measurand Name
The survey sample ID and the full name of the measurand.
B
Target
The target or expected result for the survey sample.
C
Survey Results & Method Classification
The survey results submitted for the qualitative and associated quantitative measurand. The method classification the laboratory has submitted. Ensure that your method classification is correct. If the method classification information provided by us does not allow for an adequate definition of your method then contact the RCPAQAP.
D
Histogram
Histogram showing the distribution of all results. Results highlighted in dark blue are the peer group results representing the main variable in the test method. In this case, the reagent. The light blue dot (●) represents the participant’s result.
E
Scatter plot
Scatterplot showing the distribution of all results (grey dots). Results highlighted in dark blue are peer-group results representing the main variable in the test method. The light blue dot (●) represents the participant’s result. The median is represented by a dashed line (—–) within the scatter of results.
F
Method Comparison
Each report will provide a method comparison table to review the performance across all methods for the quantitative and the associated qualitative measurand. Groups with less than 4 users will not provide quantitative results with an SD or CV. Groups with less than 3 users will not provide a median or mean result.
1. Cumulative Assessment
Provides a snapshot of participant’s performance over a survey year, displaying the results returned, the expected result and the Assessment provided (review).
B Cell Subsets
1. Performance Summary
Method system used by your laboratory:
- Method Category: The method category specific to the measurand
- Method system used by your laboratory: The Method supplied by the participant associated with the method category
- No. Users: Number of participants that use the same method category
Summary of Performance:
- Tests: All measurands that are performed by the participant.
- Target: The mean value, calculated from ALL results.
- Your result: The result submitted by the participant.
- No. Users: Number of users performing the marker
- Z-score: Represents the participant’s z-score, (Lab result – Mean / SD). Z-scores of 2.0 – 3.0 are highlighted (amber) for review. Z-scores greater than 3.0 are highlighted (red) for further action.
Overall Performance:
- Will list any result with a z-score greater than 2.0 for follow up.
2. Result Review
Provides a scatter of all results for all returned markers in a survey sample. Results highlighted in dark blue represent the participant’s peer group and the light blue dot (●) represents the participant’s result.
3. Statistical review
The statistical review provides an overview of the performance of each marker, which includes the number of users that tested the marker, the median, mean, standard deviation (SD) and Coefficient of Variation (CV).
4. Cumulative Assessment
Provides the assessments over the survey year by listing the z-scores for each measurand tested by the participant.
Myositis Specific and Scleroderma Antibodies, Liver Antibodies
1. Summary of Performance – Method system used by your laboratory
- Measurand: Lists all antibodies performed by the participant.
- Reagent: The reagent used by the participant, specific to the antibody.
- Analytical principle: The analytical principle used by the method.
- Measurement System: The measurement system (instrument) used by the participant.
Performance Assessment: Provides a review of all antibodies tested by all participants.
- Antibody: Lists all antibodies surveyed by participants.
- Your Result: Lists the results submitted by the participant, if performed.
- Expected Result: The expected result for each antibody. Results that have less than 6 users are not assessed and will list no target or assessment.
- nPart: Number of participants that provided a result for the antibody.
Overall Performance: Provides an overview of the performance, listing any antibody that did not return the expected result.
2. Result Review
The result review page contains a stacked bar chart that displays the performance of the entire list of antibodies that was returned for the survey. The legend below displays the interpretive responses returned with the target for each antibody listed on the right side of the bar chart. Results highlighted as a light blue dot (●) represents the participant’s result.
3. Performance – Specificity / Blot Intensity
 |
 |
Provides an overview of participant responses for each antibody, including the number of results returned for each antibody, the interpretive response provided and an overview of the blot intensity in a separate table.
4. Cumulative Assessment
Provides a snapshot of the participant’s performance over a survey year, displaying the results returned, the expected result and the Assessment provided.