Stage 2: Running the survey

Including the perspective of all hospital staff provides the most complete picture of the culture of the hospital. A-HSOPS 2.0 can be completed by all types of hospital staff - from housekeeping and security to nurses and hospital administrators.

The project team should also consider the level of cultural maturity and the readiness for change. Undertaking a survey of patient safety culture in a unit, department or hospital already identified as having very low levels of trust, accountability and other negative cultures needs to be done with care. Other strategies may need to be implemented first to understand the culture and build trust before conducting a survey of patient safety culture.

The survey can be administered to all staff (a census), a proportion of all staff (a representative sample), or focus on specific units or staff positions (a targeted sample).

Census

A census approach has the following advantages and, where resources are available, is the preferred option for most hospitals as it:  

• Conveys a message of inclusiveness by giving all staff the opportunity to provide feedback
• Results in data that are more representative of your hospital’s workforce
• Allows for more detailed analysis and reporting of results at a unit and department level
• Makes communication about the project easier. 

Targeted sample

A safety culture measurement project may be focused on specific groups within the hospital. This may be due to resource constraints or to examine an issue that has been identified through other means. You may focus on:

• All staff from a number of units/work areas/ departments
• All staff within specific positions or levels of experience
• A combination of the above.

This approach should never involve purposeful selection of individuals from within these groups.

A targeted approach can have the following challenges that will need to be managed through careful communication and engagement:

• Concerns about why a the focus is on a particular group
• Clarity about who is included in the sample.

Additional care  also needs to be taken to ensure that the data is reported in a way that protects the confidentiality of respondents (See the technical specifications for the A-HSOPS 2.0). 

Representative sample 

Using a representative sample is a way of decreasing the resources required to measure patient safety culture which can be useful in large hospitals.

You can expect to receive completed surveys from about 30 to 50 percent of the sample. The table below shows the recommended minimum sample sizes using a 50 percent response rate. You should update the estimated response rate based on your previous experiences with staff surveys to calculate your sample. The more staff surveyed, the more likely you are to have a representative sample. 

It is important to use a method that ensures the sample is representative – staff should never be purposefully selected to complete the survey. Options for sampling include:  

• Simple random sampling: involves selecting staff randomly so that each staff member has an equal chance of being selected
• Systematic sampling: involves selecting every Nth person from a staff list. To begin, sort your staff list by unit/work area and then by staff position. If you have a list of 1,200 names and need to select 600 to include in your sample, you begin at a random point on the list and then select every other name on your list. Thus, if you begin with the first person on the list, you select the 2nd, 4th, 6th, 8th, etc., staff member, compiling a total of 600 names in your sample.

Work with your HR team to compile a list of staff. Including the following information will allow you to remove duplicates, send out the survey and track response rates by key demographics:

• First and last name
• Email address
• Hospital unit/work area
• Staff position or job title.

Once you compile your list, review it to make sure the information is complete, up to date, and accurate. Make the following corrections where possible:

• Remove names of staff who appear on the list more than once
• Exclude staff who no longer work at the hospital
• Exclude staff who only work in outpatient setting, not in the hospital inpatient setting.

If you are conducting a census of all staff, no sampling is needed; your list is complete. If you are conducting a sampling approach you should now select the sample.

Key to successful implementation of the project is engagement with the board, senior executive and project champions. Take time to engage with these stakeholders early and make sure you have buy-in. These champions are particularly important in large hospitals where it is harder for the project coordinator to walk the floors and engage with staff. A range of resources have been developed to support communication and engagement.

Senior leadership

In addition to designating a senior executive sponsor as a part of the core project team (see – step 1.3 Form a project team and identify champions), broader senior executive engagement is critical. Demonstrating commitment sends a powerful message to staff that the survey is being taken seriously and that providing feedback is valued by the organisation. This will improve staff attitudes towards the survey, result in higher survey response rates and build trust that action will be taken in response to the findings.

Actions senior leadership can take to show their commitment to the improvement project include:

• Allocate sufficient resources including time for staff to complete the survey during work hours 
• Publicly endorse and promote the survey, via all-staff briefings, emails and memos, newsletters and intranet communication
• Direct managers to support staff participation by discussing it and facilitating time to complete it
• Tell staff why their participation is important and how results will be used
• Thank staff for participating when data collection has finished
• Ensure ongoing communication with staff through the action planning and improvement stages of the project.

Project champions

Achieving buy-in from managers and clinical leaders at all levels is also important. Communication should be early and ongoing. You will need to communicate the following to achieve buy-in:

• Why the hospital is considering conducting the survey
• What will be measured
• How the findings will be fed back to staff
• How the findings will be used for improvement.

Response rates from medical staff are often lower than other hospital staff. Engaging senior medical leadership from the onset can help. Targeted communication should focus on ways in which the project will benefit medical staff to encourage their participation in the process.

The survey was developed to be general enough for use in most hospitals. We recommend making only those changes to the survey that are absolutely necessary.

The A-HSOPS 2.0 technical specification outlines the changes that can be made to the survey. In short, changes to the survey should be limited to:

• Changes to the unit/work area and position questions to ensure that the structure reflects the hospitals structure and analysis requirements
• The addition of a small number of questions at the end of the core survey questions
• The addition of a small number of demographic questions to allow for more detailed analysis.

The updated survey should be tested with staff; the total time to complete the survey should not exceed 15 minutes.

Further guidance is provided in the the technical specifications for the A-HSOPS 2.0.

The survey should be entered into the online platform at least two weeks before the planned release date to allow time for user testing.

The importance of user testing should not be underestimated – if staff are unable to access the survey on their first attempt they are unlikely to try to access it again.  Testing is particularly important if an external vendor will send the invitations. You may need to work with your IT team to make sure emails from the vendor are not blocked.

You should test that:

• All questions are entered correctly
• The layout is user friendly on a small laptop computer and mobile phone
• The unit/work area and position response options make sense to your hospital staff
• The survey platform is accessible on hospital computers (not blocked)
• The survey platform allows multiple responses using the same IP address
• The survey invitations go to staff members inboxes (not flagged as junk) 
• The IT system can handle bulk emails to all staff (if not, you may need to send batches)
• The data file exported from the survey platform displays as expected (no skipped questions or out of range values).

Some hospitals will have the capability to pre-fill some demographic information by connecting the survey to their HR systems. This approach should only be used by hospitals where there is sufficient trust from staff that they will not be identified and that their confidentiality will be maintained. The A-HSOPS 2.0 technical specification provides information on how this can be done.  

It is useful to develop your communication materials ahead of time. The Commission has developed a number of communication templates which can be modified to meet hospitals' needs. The templates include:

• Posters 
• Flyers 
• FAQs 
• Draft communication from management 
• Draft invitations to complete the survey. 

These templates will be added to overtime – please send any additional materials you develop and would like to share to indicators@safetyandquality.gov.au.   

A range of approaches to encourage involvement in the survey have been identified through the pilot process. It is not necessary to use all of these approaches. Hospitals should choose approaches that meet their needs and suit their context.

Approaches to maximising response rates include:

• Walking the floors to encourage participation and answer questions about the project
• Ensuring it is clearly communicated that staff can complete the survey during work hours and providing dedicated times to do so
• Advertising the survey on the intranet
• Attaching the link and QR code to shared computers so the survey can be completed at an opportune time
• Updating the promotional materials with photos of actual hospital staff
• Regularly changing promotional material throughout the data collection period to maintain interest
• Customised communication for different staff cohorts explaining why it is important for them and their work
• Presenting information on the project at multiple forums – such as all staff forum, team briefings
• Connecting the project with another initiative or day – such as international patient safety day
• Hosting afternoon teas where computers or tablets are available to complete the survey  
• Encouraging competition between units/ departments on response rates
• Provide daily or live updates on the rate of completion at a unit/ work area level
• Providing specific times and spaces where staff can complete the survey
• Offering a prize or reward for participation at an individual or unit level.

The email or letter should describe the objectives of the project and the instructions for completing the survey. This should be on official letterhead/email and be sent 1-2 weeks before the survey is released.

Other pre-survey promotion may include:

• An email from senior leadership showing support for the project
• Posters highlighting the survey is coming soon
• Communication about the project through other forums such as newsletters and the intranet.

The survey link should be sent in an email early on the day of the survey. Ensure that the systems have been thoroughly tested prior to the release of the survey (see step 2.6 on this page).

On-the-day promotion of the survey may include:

• An event – for example linking to an awareness day
• An afternoon tea and presentation on the project
• Access to a staff member who can answer questions about the project.

Response rates should be tracked throughout the survey period. Many online platforms will have options to track responses automatically.  

It is useful to have a HR listing of staff members eligible for participation broken down by unit/work area to monitor response rates and target follow-up. This information is also valuable for assessing the representativeness of the survey sample.

Weekly reminder emails should be sent to encourage participation. Some platforms will allow you to target reminders to people who have not yet completed the survey. An option to unsubscribe from reminders should be included. 

See step 2.8 on this page for more approaches to maximise response rates. 

It is key that the contribution of staff is acknowledged and they are informed of the next steps of the project. This is an easy step to overlook as you move on to the work of analysing the results but it is important to take a moment to thank staff for participating and the project team for their support. This is also an opportunity to assure staff that action will be taken as a result of their efforts.