Quantitative microbial risk assessment (QMRA) is being increasingly used to support decision-making for food safety issues. Decision-makers need to know whether these QMRA results can be trusted, especially when urgent and important decisions have to be made. This can be achieved by setting up a quality assurance (QA) framework for QMRA. A Belgian risk assessment project (the METZOON project) aiming to assess the risk of human salmonellosis due to the consumption of fresh minced pork meat was used as a case study to develop and implement QA methods for the evaluation of the quality of input data, expert opinion, model assumptions, and the quality of the QMRA model (the METZOON model).
The first part of this thesis consists of a literature review of available QA methods of interest in QMRA (chapter 2). In the next experimental part, different QA methods were applied to the METZOON model.
A structured expert elicitation study (chapter 4) was set up to fill in missing parameters for the METZOON model. Judgements of experts were used to derive subjective probability density functions (PDFs) to quantify the uncertainty on the model input parameters. The elicitation was based on Cookes classical model (Cooke, 1991) which aims to achieve a rational consensus about the elicitation protocol and allowed comparing different weighting schemes for the aggregation of the experts PDFs. Unique to this method was the fact that the performance of experts as probability assessors was measured by the experts ability to correctly and precisely provide estimates for a set of seed variables (=variables from the experts area of expertise for which the true values were known to the analyst). The weighting scheme using the experts performance on a set of calibration variables was chosen to obtain the combined uncertainty distributions of lacking parameters for the METZOON model.
A novel method for the assessment of data quality, known as the NUSAP (Numeral Unit Spread Assessment Pedigree) system (chapter 5) was tested to screen the quality of the METZOON input parameters. First, an inventory with the essential characteristics of parameters including the source of information, the sampling methodology and distributional characteristics was established. Subsequently the quality of these parameters was evaluated and scored by experts using objective criteria (proxy, empirical basis, methodological rigour and validation). The NUSAP method allowed to debate on the quality of the parameters
within the members of the risk assessment team using a structured format. The quality evaluation was supported by graphical representations which facilitated decisions on the inclusion or exclusion of inputs into the model.
It is well known that assumptions and subjective choices can have a large impact on the output of a risk assessment. To assess the value-ladenness (degree of subjectivity) of assumptions in the METZOON model a structured approach based on the protocol by Kloprogge et al. (2005) was chosen (chapter 6). The key assumptions for the METZOON model were first identified and then evaluated by experts in a workshop using four criteria: the influence of situational limitations, the plausibility, the choice space and the agreement among peers. The quality of the assumptions was graphically represented (using kite diagrams, pedigree charts and diagnostic diagrams) and allowed to identify assumptions characterised by high degree of subjectivity and high expected influence on the model results, which can be considered as weak links in the model. The quality assessment of the assumptions was taken into account to modify parts of the METZOON model, and allows to increase the transparency in the QMRA process.
In a last application of a QA method, a quality audit checklist (Paisley, 2007) was used to critically review and score the quality of the METZOON model and to identify its strengths and weaknesses (chapter 7). A high total score (87%) was obtained by reviewing the METZOON model with the Paisley-checklist. A higher score would have been obtained if the model was subjected to external peer review, if a sensitivity analysis, validation of the model with recent data, updating/replacing expert judgement data with empirical data was carried out. It would also be advisable to repeat the NUSAP/Pedigree on the input data and assumptions of the final model. The checklist can be used in its current form to evaluate QMRA models and to support model improvements from the early phases of development up to the finalised model for internal as well as for external peer review of QMRAs.
The applied QA methods were found useful to improve the transparency in the QMRA process and to open the debate about the relevance (fitness for purpose) of a QMRA. A pragmatic approach by combining several QA methods is recommendable, as the application of one QA method often facilitates the application of another method. Many QA methods (NUSAP, structured expert judgement, checklists) are however not yet or insufficiently described in QMRA related guidelines (at EFSA and WHO level). Another limiting factor is the time and resources which need to be taken into account as well. To understand the degree of quality required from a QMRA a clear communication with the risk managers is required. It is therefore necessary to strengthen the training in QA methods and in the communication of its results. Understanding the usefulness of these QA methods could improve among the risk analysis actors when they will be tested in large number of QMRAs.
| Identifer | oai:union.ndltd.org:BICfB/oai:ETDULg:ULgetd-02012011-205244 |
| Date | 31 January 2011 |
| Creators | Boone, Idesbald |
| Contributors | Van Der Stede, Yves, Daube, Georges, Van Der Sluijs, Jeroen, Korsak Koulagenko, Nicolas, Aerts, Marc, Messens, Winy, Alban, Lis, Laitat, Martine, Linden, Annick, Mainil, Jacques, Saegerman, Claude, Desmecht, Daniel |
| Publisher | Universite de Liege |
| Source Sets | Bibliothèque interuniversitaire de la Communauté française de Belgique |
| Detected Language | English |
| Type | text |
| Format | application/pdf |
| Source | http://bictel.ulg.ac.be/ETD-db/collection/available/ULgetd-02012011-205244/ |
| Rights | unrestricted, Je certifie avoir complété et signé le contrat BICTEL/e remis par le gestionnaire facultaire. |
Page generated in 0.0014 seconds