The protocol for eliciting receptor impact models is structured around each landscape class defined within each bioregion or subregion. The fundamental question ‘How might selected receptor impact variables change under various scenarios of change for the hydrological response variables?’ is addressed by the experts. A very important step in the process therefore is the formalisation and specification of the relevant impact and response variables and the relationships among these variables (Chapter 4).
The protocol methodology asks experts to provide estimates of the chosen receptor impact variable at a range of values of the relevant hydrological response variables. It then uses statistical modelling to build a relationship between the hydrological response variables and the receptor impact variables, where the former are treated as covariates and the latter as the response of a linear model with potentially non-linear basis functions (sensu Hosack et al., 2016). This approach provides a repeatable protocol to develop these models via experts.
The elicitation session is designed to tackle a challenging problem, and it depends on the collaboration and cooperation of the experts. For this session, experts are asked to contribute their knowledge and expertise in a small group setting, although sometimes individually when only a single expert is available. The group format permits experts to confer and seek a consensus opinion when responding to presented scenarios. A group approach not only allows for the entire group to contribute but also permits the opportunity for feedback and group learning while responding to hypothetical scenarios.
The elicitations take place at receptor impact modelling workshops, which are separately held for each bioregion and subregion. The attending experts represent a wide range of expertise and experience. At the workshop, parallel sessions are run with experts grouped by expertise. The groups may range in size from a single individual up to a small number of individuals. The expert preparation for these group elicitation sessions is as follows.
The following steps were involved prior to the receptor impact modelling workshop:
- Experts are selected for the elicitation process. The choice of experts and their invitation into the process ensures that appropriate expertise is identified and included in the process. Relevant experts are contacted by staff collaborating among the Office of Water, the Bureau of Meteorology and the BA ecology discipline teams for each bioregion or subregion. Ideally, experts will have previously attended the relevant qualitative modelling workshop for each bioregion or subregion. This allows experts to take advantage of their previous exposure to and discussion about the bioregion or subregion, the relevant landscape classes and the receptors and hydrological response variables identified in the qualitative models. This continuity helps experts smoothly transition into a quantitative assessment for one or more receptor impact variables, while understanding the context given to a particular receptor impact variable by its relationship with respect to the landscape class and ecosystem. As noted above, the qualitative models help experts identify both potential direct and indirect hydrological impacts. Also, experts that have attended the qualitative modelling workshop will have had previous exposure to the scope and objectives of the Bioregional Assessment Programme, which leads to further gains in efficiency.
- Prior to the first meeting the experts receive material outlining the Programme, the nature and purpose of the receptor impact models and a description of the landscape classes, hydrological response variables and potential receptor impact variables. It also describes how the experts’ responses are to be used to develop the receptor impact model.
- The quality of the group elicitation depends on a collegial, open-minded and focused atmosphere. The experts are provided information on the group elicitation approach. In particular, the following guidelines are provided:
- Respect alternative opinions. Everyone has different points of view and experience that will be called upon over the course of the session.
- Practice patience. The problem is new and challenging. It will take some thought and work to assess and talk about, which can take time. Occasionally, points will need to be considered again (and even again).
- Ask questions of the session facilitators. Assessing expert opinion is a communication exercise, and please ask if you are unsure about what is being asked of you. We much prefer to address any confusion as it arises.
- Ask questions of your fellow experts. Do not assume that ‘you know what they mean’, even if you have worked together closely in the past. Everyone has different ways for expressing their views, and clarifying questions can help resolve where points of view are both different and similar.
- Recognise the minority opinion. Please speak up! The group elicitation approach depends on everyone participating. If you are uncertain, that is ok, this is accommodated by the elicitation approach that will be used in this session.
At the receptor impact modelling workshop, the following steps are completed before beginning the elicitation sessions:
- The experts are convened and given a brief introduction about BAs, landscape class definitions and hydrological response variable definitions. They are then encouraged to ask questions about the process.
- Before undergoing an elicitation session, experts are first educated about subjective probability, trained on heuristics and biases commonly encountered in expert elicitation exercises (O’Hagan et al., 2006).
- The elicitation procedure is presented (Section 6.2) to the experts. Practice examples are worked through until experts are comfortable with the method before beginning the elicitation.
- The experts are reminded of the group elicitation approach and its accompanying challenges and guidelines for expected group behaviour, as documented above. The experts are then split into their parallel sessions by expertise.
- Within each parallel session, the facilitator uses their experience of the process to highlight challenges experts may have and strategies to overcome them. The choice of receptor impact variables is discussed to identify any fundamental conceptual modelling modifications from the preceding qualitative modelling workshop.
Throughout each parallel elicitation session, a BA contact is made available to respond to all queries from the experts about questions of hydrology, ecology or the BA process.