Step 3.1 – Define triggers for action
Outcome
By completing this step, you will have a set of triggers for action that will help you ensure that the AHPEQS implementation and operation achieve the objectives you have defined. You will also have decided how to communicate these across your organisation.
Things to consider
Parameters
Consider the parameters that you need to monitor in your organisation to determine whether you are on track to achieve your organisation’s AHPEQS objectives. Parameters may include:
- Pre-administration attrition rate
Attrition can lead to sampling bias. The attrition rate is the proportion of eligible patients that cannot be administered the survey because- they do not consent to being sent a survey
- the organisation does not have the required information about the patient to administer the survey (for example, mobile phone number if the survey is administered by text message, or email address if administered by email).
- Response rate and completion rate
This is the proportion of eligible patients receiving the survey who respond partially or completely to the survey and return their responses to the surveying organisation. Partial responses lead to response bias in results; they may be received when- the survey is administered using pen and paper.
- the survey administration method does not make all questions compulsory
- Attrition and response rates for population segments
You may want to make sure that certain groups within your patient population are adequately represented in the responses you receive or that the relative representation of different groups is reflective of your overall patient population. If so, you could monitor attrition rates and response rates broken down by- cultural and linguistic background.
- Indigenous status
- age and gender
- department or specialty of admission
- reason for admission
- Number of respondents
This is the absolute number of people who send a completed response within a defined time frame, as well as numbers of respondents within each population group you would like to present disaggregated data about. Low numbers of responses may affect the analysis process and the ability to report AHPEQS results. - Average performance on individual questions
Consider whether you would like to set and communicate an expectation about how your organisation (or different parts of it) performs on each question. You could do this by- setting a goal for the average of all responses returned within a defined time frame, across all questions (this will require setting up a scoring system beforehand – see Step 3.2).
- setting a goal for the proportion of respondents selecting a particular response(s) (for example, X% select ‘always’; X+15% select ‘always’ or ‘mostly’)
- Average performance on overall (final) question
Consider whether you would like to set and communicate an expectation about how your organisation (or different parts of it) performs on the overall (final) question. You could do this by setting a goal for the proportion of respondents selecting a particular response(s) (for example, X% select ‘very good’; X+10% select ‘very good’ or ‘good’).
Baseline
If you plan to conduct a staged implementation involving a pilot study, you can use the pilot to determine the baseline measure for each of the above parameters. This helps you to decide what is achievable in your organisation in the short, medium and long term.
Note that after analysis of an initial pilot, there is an opportunity to make changes to your implementation to improve scores on each parameter. If, after a trial of the new method, scores have improved and you decide to take the new method into your full rollout, the baseline scores will now be the results from the trial of the new method.
Active consideration of AHPEQS results and response ‘red flags’
Step 3.4 will look at ways in which AHPEQS results can be integrated into workflows to ensure they are actively considered by relevant staff members. You may wish to monitor how often this happens when there are certain types of responses or trends in responses.
Consider whether you will establish red flag triggers for any instance of a particular response. For example, you may set up an alert for any positive responses to the question ‘I experienced unexpected harm or distress’, or an alert only when this is accompanied by a subsequent response that staff did not discuss it with the patient.
Triggers for action
You can set absolute and/or relative scores on each parameter to define when corrective action will need to be taken:
- Triggers based on parameter value
Consider the minimum acceptable value for each parameter (that is, the point at which you think corrective action will need to be taken). For example:- when absolute numbers of responses for a given population group pose an unacceptable risk of identifying individuals if reported.
- when attrition or response rates fall below X% of eligible discharges
- Triggers based on changes in parameter value
Consider the minimum acceptable change in value for each parameter over a defined time frame, below which corrective action will need to be taken. This type of trigger will need to take into account ceiling effects (so as not to create a trigger when the values are above a certain level to start with). For example:- when improvement for a particular population group falls below X%.
- when improvement on a specific question which was previously identified as a concern falls below X%
- when improvement on the value for the overall (final) question falls below X% in a defined time frame.
Automating the triggers
Automating triggers will make the process more useful and cost-effective for your organisation. Think about:
- How you can build automated alerts into your surveying system when trigger thresholds are reached
- How the trigger alert will be delivered and when
- Who the alert will be delivered to and who is expected to take action
- How the appropriate action is coupled with delivery of the alert
- Whether there is a need for different ‘severities’ of alert – such as a traffic light system where ‘red’ indicates immediate action required and ‘amber’ indicates action required within a particular time frame.
Defining what action will be taken for each trigger
Consider how your organisation will respond if a trigger for action occurs. An example is given below, which could be worked through for each of the triggers you have identified.
| If this trigger occurs | … this action will follow (examples only) |
|---|---|
| High pre-administration attrition rates |
Identify the problem(s), for example:
Investigate the cause(s) of the problem:
Modify the survey process based on your identification and investigation of the problem. For example:
Monitor the impact of modification in the next round of results. |
Communicating expectations
Consider who you will consult and communicate with about establishing trigger thresholds, and how you will do this. For example:
- Staff responsible for sending out the survey and gathering responses will need to be consulted about the trigger thresholds and associated actions for attrition and response rates
- Staff responsible for analysing and reporting on completed responses will need to be consulted about trigger thresholds for absolute numbers of responses and proportion of completed responses by population segment
- Frontline professionals will need to be consulted about the trigger thresholds for performance on particular questions and the survey as a whole
- Senior managers will need to be consulted about appropriate actions to take when trigger thresholds are reached.
Reviewing parameters and triggers
Think about how often you will review and modify the continued appropriateness of each trigger threshold and the types of parameters monitored.