Final Reports

RCPAQAP Biosecurity issues a Final Report for programs (e.g. Simulated Exercise Program) that have not yet transitioned to the new software platform. These contain the results of all surveys within the module, an assessment of the results and extensive educational material.

Survey Reports

This report follows a standard structure adopted by other RCPAQAP programs and replaces the previous preliminary and final reports.

The structure of the survey report provides individual participants with a summary of their performance, a review of individual performance, a discussion of survey results with commentary for all samples and a cumulative performance. All Biosecurity reports will generally follow the same structure:

1. Summary of Performance – The summary of performance provides a snapshot of participation and assessment in the program.

1. Summary of Performance for specimen programs – This includes the Bacterial, Environmental and Viral Specimen programs. This section shows a table for the performance assessment that includes ‘your result’, the ‘target/expected result’ and the ‘assessment’ for each sample. It also includes the overall performance of participant results, with an indication of the samples that require review if a result does not match the target result.
   An example of this for the report issued for the Viral Specimen program can be seen below.

   Summary of Performance

   Performance Assessment
   	Sample: BS-VSM-23-04
   Test	Your Result	Expected Result	Assessment
   Ebolavirus final interpretation	Negative	Negative	Concordant
   Marburgvirus final interpretation	Negative	Negative	Concordant
   Lassavirus final interpretation	Positive	Positive	Concordant
   Rift Valley fever virus final interpretation	Negative	Negative	Concordant
   Crimean Congo haemorrhagic fever virus final interpretation	Negative	Negative	Concordant
   	Sample: BS-VSM-23-05
   Test	Your Result	Expected Result	Assessment
   Ebolavirus final interpretation	Negative	Negative	Concordant
   Marburgvirus final interpretation	Negative	Negative	Concordant
   Lassavirus final interpretation	Negative	Negative	Concordant
   Rift Valley fever virus final interpretation	Positive	Positive	Concordant
   Crimean Congo haemorrhagic fever virus final interpretation	Negative	Negative	Concordant
   	Sample: BS-VSM-23-06
   Test	Your Result	Expected Result	Assessment
   Ebolavirus final interpretation	Negative	Negative	Concordant
   Marburgvirus final interpretation	Negative	Negative	Concordant
   Lassavirus final interpretation	Negative	Negative	Concordant
   Rift Valley fever virus final interpretation	Negative	Negative	Concordant
   Crimean Congo haemorrhagic fever virus final interpretation	Negative	Negative	Concordant

   Overall Performance All results returned match expected results.

2. Summary of Performance for Whole Genome Sequencing and Mycobacterial Specimen program –In addition to what is included in the survey reports for specimen programs (see above), the report for this program has additional information, including the assessment criteria for the submitted sequence data, the assessment of the sequencing quality of each sample and other areas relevant to the organism assessed (e.g. variant calling).  An example of this for the report issued for the Whole Genome Sequencing program can be seen below.An example of this for the report issued for the Mycobacterial Specimen program can be seen below:

   

c. Summary of Performance for Emerging Biological Threats program – In addition to what is included in the survey reports for specimen programs (see above), the report for this program has a table for the ‘method system used by the participant’ and the ‘expected results’ for each sample.
An example of this for the report issued for the Emerging Biological Threats program can be seen below. An example of this for the report issues for the Mycobacterial Specimen Program can be seen below:

Summary of Performance

2. Result Review – The result review section of the report displays a breakdown of the results returned from participants.

1. Result Review for specimen and Emerging Biological Threat programs – The section displays graphs with your qualitative result highlighted as a blue dot. Depending on the program, the graph may also compare your qualitative result to all other participants (grey bar) and those using the same method (dark blue overlay). A scatter graph is provided adjacent to the interpretative responses, displaying quantitative results (if supplied) of the cycle threshold (Ct)-value for organism-specific gene targets within a sample. These results are not assessed but can give an indication of performance. The program does not use statistical design or analysis to evaluate results. The results for each sample are presented on a separate page for each target, depending on the programs. The Bacterial and Environmental Specimen programs report consists of result review for the ‘Direct Sample’, ‘Culture’, ‘Molecular’, and the ‘Identification and Notification’. The Viral Specimen and Emerging Biological Threats programs report consists of the responses for the final interpretation of all pathogen-specific targets, a table showing a breakdown of all assays in use and the results submitted.
2. Result Review for Whole Genome Sequencing program – The section displays graphs and scatter plots of all responses submitted for the questionnaire from the myQAP portal, with your result highlighted as a blue dot. Depending on the type of measurand, the graph may also incorporate the assessment review of your results (e.g. concordant, discordant, minor discordant, not assessed etc.)

3. Commentary – Each report includes commentary, discussing survey results for all samples. It also provides suggestions and recommendations for follow-up investigation.

4. Cumulative assessment – It summarises the cumulative performance over the year and previous years.

Assessment of Participant results

RCPAQAP Biosecurity offers a range of specialised programs with a diverse but small participant base. As a result, there may not be sufficient results (data points) for any survey to validate the information statistically.

There is a vital research element in developing specimens for the RCPAQAP Biosecurity, and participant access to the equipment required to test these specialised specimens varies greatly. RCPAQAP Biosecurity surveys are often used to understand the performance of specific tests and equipment and the relevant inhibitions that could potentially interfere with the testing.

In general, the terms used to grade survey qualitative results are listed below:

• Concordant – Matches the target or expected result
• Discordant – Does not match the target result
• Minor Discordant – A result that may have partial agreement to the target result
• Not Assessed – The result is not within the assay capability or is beyond the scope of acceptance for grading
• No Participation – The participant failed to take part in the survey.

Discordant results will be highlighted in red and minor discordant results will be highlighted in amber.  The result that is highlighted as discordant and minor discordant will be listed in the ‘Overall Performance’ summary at the bottom of the page.

For the Mycobacterial Specimen program, the expected results for all results (excluding WGS) are determined from the consensus of all results. Results are only assessed when ≥ 80% of participants return the same value and when there are 6 or more results.

• If a test achieves ≥ 80% consensus/agreement from 6 or more results, this is then listed as the expected value. Participants who return the expected value are assessed as “CONCORDANT”
• If a test achieves ≥ 80% consensus from 6 or more results, participants in the minority group are assessed as inconsistent with the consensus and their results are listed as “DISCORDANT”
• When < 80% or < 6 participants agree, the uncertainty of the results increases, so no assessment is made and all results are reported as “NOT ASSESSED”.