Development of the Australian Hospital Survey on Patient Safety Culture
An overview of the identification, selection, modification and testing of the survey.
Expert advice
The Safety Culture Measurement Expert Advisory Group was convened to support the development of the patient safety measurement toolkit. Members were nominated to represent a broad range of perspectives on patient safety culture and for their relevant expertise in safety culture. Consideration was also given to ensuring representation from a range of states and territories. The group provided advice throughout the development of the project. Membership is provided at the end of this page.
Information and evidence gathering
The Commission collected evidence and information from a range of sources, including:
- Environment scan and literature review – Patient safety culture in healthcare organisations: concepts, tools and impacts
- Review – Safety Culture Assessment in Health Care: A review of the literature on safety culture assessment tools
- Consultation – Survey of hospital staff
- Consultation – Interviews with public and private hospital executives
- Mapping – of the domains measured by state and territory organisational and engagement surveys.
Shortlisting candidate surveys
Given the substantial work undertaken to define and measure patient safety culture internationally and the number of validated surveys available, the project focused on identifying and modifying an existing survey rather than developing a new survey.
The evidence search identified a large number of surveys measuring patient safety culture. A candidate list of surveys was identified that were considered the most suitable to evaluate patient safety culture within hospitals, with potential for large-scale implementation. All of these surveys have been validated and are being used in Australia or overseas. The Candidate surveys were:
- Hospital Survey on Patient Safety Culture, Version 2 (HSOPS 2.0)
- Safety Attitudes Questionnaire
- Safety, Communication, Operational Reliability and Engagement
- Canadian patient safety climate survey.
Assessment of the candidate surveys against the selection criteria
The candidate surveys were assessed against a criteria agreed by the advisory group. The criteria was developed to provide structure for the process and to help balance competing priorities in decision making. The criteria assessed the extent to which the surveys:
- Measured meaningful aspects of safety culture
- Were feasible for implementation in Australian hospitals
- Were psychometrically valid.
The advisory group identified one survey with the potential for modification and testing in Australia – the HSOPS 2.0 developed by the Agency for Healthcare Research and Quality. The original version of this survey was released in 2004 and has been used widely internationally. An updated version of the survey was released in 2019 after testing in 25 hospitals in the US. The advisory group advised this survey had the best coverage of the priority areas for measurement and the most contemporary view of patient safety culture.
Modification and pilot testing
The advisory group supported the Commission to modify the HSOPS 2.0 to ensure that the language used in the survey was appropriate for an Australian setting. Cognitive testing was undertaken with 20 hospital staff to understand how the questions were understood by a range of hospital staff and to identify additional changes to improve understanding.
Pilot testing was required to validate the survey in an Australian hospital setting and identify if the survey could be shortened.
The Commission sought expressions of interests to participate in the pilot from public and private hospitals across Australia. Nine public and private hospitals participated in the pilot.
Pilot results
Information from the pilot along with advice from the advisory group was used to test the survey and identify items that could be removed. The final results of the analysis using a 26 item model yielded an adequate to good fit for the confirmatory factor analysis indices (Tucker Lewis Index = 0.938, Comparative Fit Index = 0.950, Root Mean Square Error of Approximation = 0.050).
Reliability was assessed for the nine composites. All composites apart from the handover and information exchange composite met accepted cut-offs for reliability. These results were similar to those found in the US pilot.
While the handover and information exchange composite did not meet the cut off for composite reliability, the items were retained due to the importance of measurement and improvement in this area. Further work will be undertaken to improve these items when the survey is reviewed.
| Composite | No. items | α |
|---|---|---|
| Supervisor or manager support for patient safety | 3 | 0.800 |
| Teamwork | 3 | 0.754 |
| Communication openness | 3 | 0.808 |
| Reporting patient safety events | 2 | 0.734 |
| Organisational learning - continuous improvement | 3 | 0.778 |
| Communications about error | 3 | 0.863 |
| Hospital management support for patient safety | 3 | 0.799 |
| Response to error | 3 | 0.816 |
| Handovers and information exchange | 3 | 0.679 |
Note: Cronbach’s Alpha values greater than 0.70 are considered as satisfactory, and 0.80 as excellent.
Safety Culture Measurement Expert Advisory Group Membership
| Member name | Speciality, organisation, state and role |
|---|---|
| A/ Prof Amanda Walker | Specialist in palliative medicine, NSW, Clinical Director, the Commission, Chair |
| Ms Cathy Jones | Chief Quality and Risk Officer, TLC Healthcare, National, Private Hospital Sector Committee representative |
| Ms Claire Hewat | Dietician, CEO, Allied Health Professions Australia, National, Allied Health Professions Australia representative |
| Ms Cindy Schultz-Ferguson | Consumer Representative, VIC, Individual representative |
| Prof Deborah Bailey | Director of Surgery, Gold Coast Hospital, QLD, Royal Australasian College of Surgeons representative |
| Dr Genevieve Gabb | Senior Staff Specialist, General Medicine and Cardiology, SA, The Royal Australasian College of Physicians representative |
| Ms Heni Hongara | Clinical Nurse Manager, Royal Darwin Hospital, NT, Individual representative |
| Ms Jessica Moran | Manager Clinical Redesign, Central Coast Local Health District, NSW, The Australian College of Midwives representative |
| Dr Jodi Johnson-Glading | Deputy Chief Medical Officer, Health Professional Policy and Advisory Services Group, Department of Health and Human Services, TAS, Inter-Jurisdictional representative |
| Dr Jonathan Tang | Advanced Paediatric Registrar, Women’s and Children’s Hospital Network, Department of Health, SA, Individual representative |
| Dr Kate Churruca | Research Fellow, Australian Institute of Health Innovation, Macquarie University, NSW, Individual representative |
| Dr Louise Ellis | Research Fellow, Australian Institute of Health Innovation, Macquarie University, NSW, Proxy for Kate Churruca, Individual representative |
| Ms Melanie Robinson | Nurse, CEO, Congress of Aboriginal and Torres Strait Islander Nurses and Midwives, ACT, Congress of Aboriginal and Torres Strait Islander Nurses and Midwives representative |
| Ms Natasha Thompson | Executive Director, People and Analytics Branch, Victorian Public Service Commission, VIC, Individual representative |
| Dr Paul Miles | Program Manager, Digital Patient Safety and My Health Record in Emergency Departments, The Commission, National |
| Ms Prachi Javalekar | Safety Culture Coordinator, Western Sydney Local Health District, NSW, Australian College of Nursing representative |
| Dr Suellen Allen | Director, Clinical Communications, Mental Health and Cognitive Impairment, the Commission, National |