Assessment of Diagnostic Proficiency Programs
The RCPAQAP diagnostic programs include General, Breast, Oral and Maxillofacial, Dermatopathology, Mohs, Forensic, Gynaecology, Paediatric, Urology, Neuropathology, Gastrointestinal, Electron Microscopy, Bone and Soft Tissue, Liver and Pancreaticobiliary, Endocrine, Renal (Medical), Head and Neck, Haematopathology and Thoracic.
A Preliminary Report comprising participant submitted preferred diagnosis and preliminary target diagnosis for each case is available within two days after the survey close date. Following analysis of all responses (6-8 weeks after survey closing date), participants are provided with a Survey Report comprising assessment results, case performance analysis and discussion.
Assessments are assigned according to the preferred diagnosis for each case against the “target” diagnosis as follows:
Concordant | The preferred diagnosis is essentially identical to the target diagnosis |
Minor discordance | The preferred diagnosis has one or more minor differences from the target |
Differential diagnosis | The target diagnosis is included as part of a list of potential diagnoses |
Discordant | The preferred diagnosis is substantially different from the target diagnosis |
No participation | No result submitted |
Unable to be assessed | Unable to be assessed |
Samples with a discordant rate of 20% or greater are reviewed by the appropriate advisory committee for consideration of assigning the sample as educational or void for the purposes of diagnostic proficiency testing. Review outcomes will be recorded in the Survey Report.
All reports are available in myQAP on the Reports tab.
Assessment of HER2 Brightfield ISH (BRISH) Breast Diagnostic Program
The HER2 BRISH breast diagnostic program comprises virtual images for interpretation of dual brightfield ISH probe staining.
HER2 IHC score
The HER2 IHC score assessments are assigned according to the score submitted for each case against the target score as follows:
Concordant | The submitted HER2 IHC score is consistent with the target |
Minor discordance | The submitted HER2 IHC score deviates from the target |
Discordant | The submitted HER2 IHC score is substantially different from the target |
HER2 ISH status
The overall HER2 ISH status (Amplified, Non-amplified or Uninterpretable with comment) assessments are assigned according to the status submitted for each case against the target score as follows:
Concordant | The submitted HER2 ISH status is consistent with the target |
Discordant | The submitted HER2 ISH status deviates from the target |
The expected Mean HER2 gene count per cell result is calculated from the median of all results submitted. For this median to be accurate, participants are requested to submit numeric results only, not < or > results (e.g. >10). This is not assessable. A scatter plot is provided to illustrate inter-observability and extreme outlier results. Participants are requested to review their method of counting if self-identified as an outlier.
The expected Mean CEP17 count per cell result is calculated from the median of all results submitted. For this median to be accurate, participants are requested to submit numeric results only, not < or > results (e.g. >10). This is not assessable. A scatter plot is provided to illustrate the inter-observability and extreme outlier results. Participants are requested to review their method of counting if self-identified as an outlier.
The expected HER/CEP17 ratio result is calculated from the median of all results submitted. For this median to be accurate, participants are requested to submit numeric results only, not < or > results (e.g. <2). This is not assessable. A scatter plot is provided to illustrate the inter-observability and extreme outlier results. Participants are requested to review their method of counting if self-identified as an outlier.
HER2 ISH ASCO-CAP Group
The HER2 ISH ASCO-CAP Groups are applied using both the HER2 IHC score and HER2 ISH status findings as per the RCPA approved guidelines ASCO CAP 2018 HER2 Testing for Breast Cancer Guidelines – Recommendations for Practice in Australasia tabulated below.
Group |
Biology |
HER2/CEP17 Ratio |
Mean HER2 copy number |
2018 ASCO CAP Recommendation |
1 | Classic HER2 amplified cancer | ≥2.0 | ≥4.0 | Positive |
2 | Monosomy 17 | ≥2.0 | <4.0 | Negative, unless concurrent IHC3+ |
3 | Co-amplification, previously polysomy 17 | <2.0 | ≥6.0 | Negative, unless concurrent IHC 2+ or 3+ |
4 | Borderline/\equivocal | <2.0 | ≥4.0 and <6.0 | Negative, unless concurrent IHC 3+ |
5 | Classic HER2 non-amplified cancer | <2.0 | <4.0 | Negative |
Assessments are assigned according to the group submitted for each case against the target HER2 ISH ASCO-CAP group as follows:
Concordant | The submitted HER2 evaluation is consistent with the target |
Minor discordance | The submitted HER2 evaluation deviates from the target |
Discordant | The submitted HER2 evaluation is substantially different from the target |
Assessment of HER2 Brightfield ISH (BRISH) Gastric Diagnostic Program
The HER2 BRISH gastric diagnostic program comprises virtual images for interpretation of brightfield ISH probe staining.
HER2 IHC score
The HER2 IHC score assessments are assigned according to the score submitted for each case against the target score as follows:
Concordant | The submitted HER2 IHC score is consistent with the target |
Minor discordance | The submitted HER2 IHC score deviates from the target |
Discordant | The submitted HER2 IHC score is substantially different from the target |
HER2 ISH status
Data Analysis and Assessment Criteria Handbook Page 59 The overall HER2 ISH status (Amplified, Non-amplified or Uninterpretable with comment) assessments are assigned according to the status submitted for each case against the target score as follows:
Concordant | The submitted HER2 ISH status is consistent with the target |
Discordant | The submitted HER2 ISH status deviates from the target |
The expected Mean HER2 gene count per cell result is calculated from the median of all results submitted. For this median to be accurate, participants are requested to submit numeric results only, not < or > results (e.g. >10). This is not assessable. A scatter plot is provided to illustrate inter-observability and extreme outlier results. Participants are requested to review their method of counting if self-identified as an outlier.
The expected Mean CEP17 count per cell result is calculated from the median of all results submitted. For this median to be accurate, participants are requested to submit numeric results only, not < or > results (e.g. >10). This is not assessable. A scatter plot is provided to illustrate the inter-observability and extreme outlier results. Participants are requested to review their method of counting if self-identified as an outlier.
The expected HER/CEP17 ratio result is calculated from the median of all results submitted. For this median to be accurate, participants are requested to submit numeric results only, not < or > results (e.g. <2). This is not assessable. A scatter plot is provided to illustrate the inter-observability and extreme outlier results. Participants are requested to review their method of counting if self-identified as an outlier.
Assessment of the interpretation of the HER2 IHC and HER2 BRISH Gastric adenocarcinoma cases is determined using the updated 2016 HER2 Testing and Clinical Decision Making in Gastroesophageal Adenocarcinoma: Guideline from the College of American Pathologists, American Society for Clinical Pathology, and the American Society of Clinical Oncology.
Immunohistochemistry Breast Marker Audit
The Breast Marker Audit is offered as a separate program. This audit is open for two months to allow participants adequate time to submit data. There is no technical component to this survey, but it comprises online submission of the hormone receptor and HER2 status for each case of early invasive primary (not stage 4 disease) breast carcinoma. It is recommended that 100 cases from the preceding year are submitted.
This exercise aims to review the reporting of immunohistochemistry (IHC) and in-situ hybridization (ISH) breast markers through an audit of clinical results. The exercise is designed to assist participants in assessing the quality of oestrogen, progesterone and HER2 receptor reporting within their laboratories. This is an important quality assessment activity, established in response to ongoing difficulties experienced by laboratories in this area of testing.
The RCPAQAP will assess performance in the breast marker audit according to the following criteria and table below:
Concordant | Proportion of ER positive, ER-/PR+ and HER2 ISH amplified results are within the ranges specified |
Minor discordance | Proportion of ER positive, ER-/PR+ or HER2 ISH amplified results are outside the ranges specified |
Discordant | Proportion of ER-/PR+ results are >7% |
Unable to be assessed |
Not relevant, less than 50 tests or late submission |
|
Expected range (%) |
Acceptable range (%) |
Minor discordance range (%) |
Discordant range (%) |
Percentage ER positive tests (of total ER tests) | 79-87 | 77-89 | <77 or >89 | N/A |
Percentage ER negative and PR positive tests (of total PR tests) | <5 | N/A | 5-7 | >7 |
Percentage HER2 ISH amplified | 11-16 | ≥8 and <21 | <8 or ≥21 | N/A |
Note: HER2 IHC results are NOT included in the assessment of concordance due to the availability of HER2 ISH to confirm or clarify the IHC results.
References:
- ASCO CAP 2018 HER2 Testing for Breast Cancer Guidelines – Recommendations for Practice in Australasia.
- ASCO-CAP – HER2 Testing in Breast Cancer – 2018 Focused Update.
- ASCO 2022: Practice-Changing Findings Identify HER2-Low as a Targetable Subset of Breast Cancer, Redefining Treatment for More Than 60 Percent of HER2-Negative Metastatic Breast Cancer Patients
Assessment of Technical Proficiency Programs
The RCPAQAP Technical Proficiency programs include General, Frozen Section and Neuropathology.
Following assessment of submissions (8-10 weeks after the survey closing date), participants are provided with a Survey Report containing assessment results, which comprise the exercise analysis and discussion.
Criteria for assessment may vary for each survey depending on the type of exercise or stain and these are included in the survey instructions. As a guide, an example of the criteria for H&E staining is provided below:
Staining quality
- The effectiveness in demonstrating nuclear membranes, nucleoli, chromatin of vesicular and hyperchromatic nuclei
- Definition of fine and coarse chromatin
- Absence of carryover of haematoxylin into the cytoplasm
- The effectiveness in demonstrating all non-nuclear material e.g. cytoplasm, fine and dense connective tissue fibres, skeletal and smooth muscle and red blood cells (where present)
- Definition of muscle fibres of blood vessels
- Definition of kidney basement membrane
- Uniformity of staining across slide
- Absence of contaminants
Section presentation
- Coverslip placed centrally over entire section and within boundaries of slide
- Absence of excess mountant
- Absence of bubbles
- Absence of artefacts from dehydration, clearing and mounting
Four members of the Technical Advisory Committee will independently score each slide out of 5. Where there is a discrepancy of ≥2 marks between assessors, the submitted slide is reviewed. The assessor marks are then reported as an average.
Scoring of slides
0 | No staining/ incorrect antibody applied to slide |
1 | Undiagnosable, very poor staining and none of the assessment criteria met |
2 | Unsatisfactory staining – criteria have not been met, diagnosis may be impacted upon |
3 |
Criteria has been met at a basic level |
4 | Above average |
5 | Reflects a perfect or nearly perfect fulfilment of the criteria |
Assessment Classification Marks
Satisfactory | A mark equal to or greater than 3.0 |
Borderline | A mark greater than or equal to 2.5 and less than 3.0 |
Unsatisfactory | A mark of less than 2.5 |
Unable to be assessed |
The participant indicated that the exercise was not relevant, or the submission was late or unable to be assessed (e.g. broken slide) |
For the special staining exercise, participants are required to mark the area of interest on the control section before submission. Participants are required to submit their routine control section which will be assessed by the Technical Advisory Committee as either ‘Satisfactory’ or ‘Unsatisfactory’. Failure to submit a control will be recorded as ‘No submission received’. Note that the assessment result of the control section (IQC) is not an indication of your EQA on the provided test section.
Assessment of Immunohistochemistry Programs
The RCPAQAP Immunohistochemistry programs include: General Markers, Breast Markers, Brightfield ISH (BRISH) for Gastric and Breast cancers and Epstein-Barr virus encoded RNA (EBER), Lymphoma Markers, Immunohistochemistry Mismatch repair (MMR) proteins, Neuropathology and Immunohistochemistry PD-L1.
Following assessment of submissions (8-10 weeks after the survey closing date), participants are provided with a Survey Report consisting of assessment results, which comprise the exercise analysis and discussion.
Criteria for assessment of immunohistochemistry staining may vary for each survey depending on the type of exercise and are included in the survey instructions.
The RCPAQAP will assess the immunohistochemical labelling according to the following criteria:
- Intensity of true positivity is of reasonable strength
- Absence of background staining (good signal to noise ratio)
- Sensitivity – all target tissues labelled
- Localisation – only target antigenic sites labelled
- Chromogen character – crisp and distinct
- Counterstain quality – complementary not obscuring
- Absence of artefacts
Four members of the Immunohistochemistry Advisory Committee will independently score each slide out of 5. Where there is a discrepancy of ≥2 marks between assessors, the submitted slide is reviewed. The assessor’s marks are then reported as an average. At PD-L1 assessments, in addition to the technical assessors, experts in the various scoring algorithms (TPS, CPS, IC) are present to provide a count against each section.
Scoring of slides
0 | No staining/ incorrect antibody applied to slide |
1 | Undiagnosable, very poor staining and none of the assessment criteria met |
2 | Unsatisfactory staining – criteria have not been met, diagnosis may be impacted upon |
3 |
Criteria has been met at a basic level |
4 | Above average |
5 | Reflects a perfect or nearly perfect fulfilment of the criteria |
Assessment Classification Marks
Satisfactory | A mark equal to or greater than 3.0 |
Borderline | A mark greater than or equal to 2.5 and less than 3.0 |
Unsatisfactory | A mark of less than 2.5 |
Unable to be assessed |
The participant indicated that the exercise was not relevant, or the submission was late or unable to be assessed (e.g. broken slide) |
For technical assessment, participants are required to return the stained slide together with their in-house controls to RCPAQAP. Controls are not assessed but will be reviewed if the test slide is assessed as unsatisfactory and comments provided.
Assessment of ALK and ROS1 Translocation in non-small cell lung carcinoma (NSCLC) program
This program consists of separate assessable modules according to the method:
- Immunohistochemistry staining for technical and interpretation proficiency. The assessment criteria for the IHC technical proficiency is the same as for the assessment of immunohistochemistry programs. The interpretation proficiency assessment is classified against the target response using the interpretation performance criteria listed below.
- Fluorescent in situ hybridization (FISH) technique interpretation proficiency. Participating laboratories are required to stain the slide using their routine FISH technique and provide method details including limits of detection, which will be used to determine the final assessment of genotyping results. This information may also be used to assist participating laboratories in troubleshooting if required.
- Genotyping: detection of an ALK or ROS1 gene rearrangement i.e. present or absent
- Interpretation: comment on the therapeutic implication of genotyping result based on the mock clinical scenario provided
- Molecular technique interpretation proficiency. Participating laboratories are required to perform tumour macrodissection, nucleic acid extraction and molecular analysis of gene fusion variant. Participants are also required to provide method details including limits of detection, which will be used to determine the final assessment of genotyping results. This information may also be used to assist participating laboratories in troubleshooting where applicable.
- Genotyping: detection of an ALK or ROS1 fusion variant i.e. present or absent
- Interpretation: comment on the therapeutic implication of genotyping result based on the mock clinical scenario provided
The RCPAQAP will assess interpretation performance according to the following criteria:
Concordant | Meets the expected response |
Minor discordance | Acceptable, however has one or more minor differences from the target |
Discordant | Unacceptable or does meet the expected response |
Unable to be assessed | The submission was unable to be interpreted |
No participation | No results submitted |
Assessment of Sarcoma Gene Testing Program
This program consists of separate assessable modules according to the method:
- Fluorescent in situ hybridisation (FISH) technique interpretation proficiency. Participating laboratories are required to stain the slide using their routine FISH technique and provide method details including scoring criteria, which will be used to determine the final assessment of FISH results. This information may also be used to assist participating laboratories in troubleshooting if required.
FISH: Detection of the gene rearrangement or amplification i.e., detected or not detected - Molecular technique interpretation proficiency. Participating laboratories are required to perform tumour macrodissection, nucleic acid extraction and molecular analysis of gene fusion variants or copy number changes. Participants are also required to provide method details that will be used to determine the final assessment of genotyping results. This information may also be used to assist participating laboratories in troubleshooting if required.
Genotyping: detection of the gene fusion variant or amplification, i.e., detected or not detected
The RCPAQAP will assess interpretation performance according to the following criteria:
Concordant | Meets the expected response |
Minor discordance | Acceptable, however has one or more minor differences from the target |
Discordant | Unacceptable or does not meet the expected response |
Unable to be assessed | The submission was unable to be interpreted |
No participation | No results submitted |
Assessment of Fluorescence in-situ Hybridisation (FISH) Tumour Diagnostic Program
Participating laboratories are required to stain their slide using their routine FISH technique and provide method details including limits of detection, which will be used to determine the final assessment of genotyping results. This information may also be used to assist participating laboratories in troubleshooting where applicable.
Genotyping: detection of the fusion variant/translocation i.e. present or absent
The RCPAQAP will assess interpretation performance according to the following criteria:
Concordant | Meets the expected response |
Minor discordance | Acceptable, however has one or more minor differences from the target |
Discordant | Unacceptable or does not meet the expected response |
Unable to be assessed | The submission was unable to be interpreted |
No participation | No results submitted |
Assessment of Transmission Electron Microscopy (TEM) Technical Program
The RCPAQAP will assess the images submitted required for the Technical exercise using a two-tiered approach
Assessment criteria according to the following criteria:
The submitted material is assessed in a two-tiered approach according to the following criteria:
- Diagnostic suitability
- The presence of the appropriate diagnostic feature(s), enabling an accurate review and diagnosis by a third party
- Submission of a sufficient number of images to fully document the diagnostic features
- Use of an adequate range of magnification to visualise diagnostic features.
- Technical quality:
- Images of sufficient number and quality, enabling an accurate review and diagnosis by a third party
- Image presentation and quality, including magnification, image size, contrast and overall density and evenness, enabling diagnostic features to be clearly visible and easily identifiable
- Images should not show substantial or consistent artefacts due to processing, sectioning or staining, or imaging (microscope) or camera artefacts.
The Advisory Committee understands that in some circumstances no specific diagnosis can be made, or that all diagnostic features may not be contained in the sample received for EM. In such a case the committee will assess the submission based on a suitable number of images representing the findings present, rather than expect specific diagnostic features to be present.
Assessment classification marks
Four members of the Electron Microscopy Advisory Committee assess each set of micrographs submitted and grade the submission separately for diagnostic suitability and technical quality, where the assessment scores are as follows:
Score: | Assessment: |
< 2.5 | Unsatisfactory |
≥ 2.5 and < 3.0 | Borderline |
≥ 3.0 | Satisfactory |
The marks are then collated between members of the panel, and the submission is assessed by consensus, on diagnostic and on technical grounds, as either satisfactory or unsatisfactory.
Satisfactory vs unsatisfactory assessment
Submissions are assessed as either satisfactory or unsatisfactory on their diagnostic and technical merits (separately) and are reported to the participating laboratory. The numerical score allocated to a submission remains confidential and is not provided. A submission may be rated as unsatisfactory or borderline due to the following reasons:
- The number of images submitted was deemed insufficient to reach a diagnosis.
- The quality of the images was very poor overall, impairing their interpretation.
- Artefacts and contaminants were present in the majority of images, interfering with the examination of the diagnostic features.
Following assessment of submissions (8-10 weeks after the survey closing date), participants are provided with a Survey Report comprising assessment results and a Generic Report comprising the exercise analysis and discussion.