Case studies of PROMs implementation

Since 2012, the Flinders Medical Centre (FMC) has been using the BREAST Q patient-reported outcome measure (PROM) to deliver patient-centred care, support clinical innovation and demonstrate the effectiveness of plastic and breast reconstructive surgeries. Patient outcomes are collected using paper-based questionnaires during the patient’s first appointment and then at about 4 weeks, 3, 6 and 12 months following their surgeries. The PROM data assesses clinical outcomes from the patient’s perspective and allows the clinician to monitor the psychosocial wellbeing of their patients and tailor clinical decisions to better address their individual needs. Aggregated data from the BREAST-Q are used to give patients insights into the differences in long-term patient satisfaction and wellbeing associated with various clinical procedures, thus empowering them to make truly informed choices on the type of care they wish to receive. PROM data is also accumulated to demonstrate the clinical effectiveness of the breast reconstruction clinic to both internal and external stakeholders. When combined with other clinically-relevant indicators, such as complication rates and hospital re-admission, the data collected using the BREAST-Q PROM are powerful in assessing new surgical techniques both from a patient and a clinical perspective. 

Buy-in and engagement of clinical and non-clinical staff was the biggest facilitator of the implementation of PROMs at the FMC, where a designated breast reconstruction nurse practitioner has coordinated and facilitated their implementation. The nurse periodically presents results from the PROM data to sustain the interest of staff, showing them the value of the additional work they do in collecting and administering the patient questionnaires. Using a robust and high-quality PROM is believed to be the main contributor to patient engagement, resulting in high participation rates among them. The lack of technological infrastructure and the absence of national or regional ethics approval pathways for approving the use of PROMs in quality improvement is the main barrier the clinic has encountered.

Recognising that traditional mortality and morbidity data alone are insufficient and fail to measure the impact of clinical practice from the patient perspective prompted Dr. Nicola Dean, the head of Breast Reconstruction and Plastic Clinic at Flinders, to pursue the implementation of PROMs in her unit. From her point of view, having meaningful measures is crucial for clinicians to see the value of the work they are doing. Metrics in health care should be designed to measure whether goals initially set for care and treatment are achieved or not; above all the patient is the person most able to determine this. The use of PROMs in the FMC reflects a paradigm change from a disease-focused health care system to one that is built around patient satisfaction and quality of life.

'I had a patient who was diabetic, obese and had a smoking history. She had a bilateral mastectomy and was desperate to have bilateral breast reconstruction. She was at a very high risk of surgical complications because of her health conditions. If you looked at it from a classical point of view, she had a readmission into hospital, a wound infection and she had some fat necrosis requiring re-operations. If you look at it from a clinical point of view, you would say that it is not an operation worth doing. But from a patient point of view, she ranked her outcome extremely highly and her physical and psychosocial wellbeing were massively increased. She had significant depression and reduced quality of life before the surgery but then she was completely transformed after the surgery. If you only look at traditional outcome metrics you don’t pick that up. It is only when you have this meaningful measure from the patient that you realise why you have done something good.'

Dr. Nicola Dean, Head of the Plastic and Reconstructive Surgery Unit, Flinders Medical Centre

Further Information

• BREAST-Q questionnaire
• Plastic and Reconstructive Surgery Services at Flinders Medical Centre
• Pusic, A. L., Klassen, A. F., Scott, A. M., Klok, J. A., Cordeiro, P. G., & Cano, S. J. (2009). Development of a new patient-reported outcome measure for breast surgery: the BREAST-Q. Plastic and reconstructive surgery, 124(2), 345-353.

Up until 2012, the number of re-operations was the primary indicator used in Australia to identify failure in surgical reconstruction or replacement for knees and hips. While important to measure, this indicator does not take into account the many subjective aspects of the post-surgical patient experience including pain, functional improvement and quality of life. To address this gap, the Arthroplasty Clinical Outcomes Registry, National (ACORN) was a multisite initiative set up to monitor, evaluate and report on clinical outcomes. For the first time in Australia, Patient Reported Outcome Measures (PROMs) for knee and hip arthroplasty were collected from multiple sites in a centralised and standardised fashion. With the broader aim of driving practice improvement and informed clinical decision making, the registry produced and delivered comprehensive reports of patient outcomes to prospective surgical patients, surgeons and hospital departments of participating institutions.

From an operational perspective, a designated site coordinator in each of the participating institutions was responsible for identifying patients meeting the inclusion criteria and explaining the purpose of ACORN to them, as well as ensuring the integrity and quality of the data collected. Using an op-out consent approach bolstered the numbers of patients in the registry. Initial data collection took place at the pre-admission clinic, with a follow-up interview with the patient then occurring approximately six months after their procedure. In 2017, the registry successfully reached a follow-up rate of almost 95%. Using phone interviews was considered to be the main facilitator of this success, as postal follow-up was shown to have a very low response rate. Within hospitals, having surgeons engaged in the process was also key to successfully implementing the collection of patient-reported outcome data.

A range of demographic and clinical data (e.g., comorbidities) were collected from patients in addition to PROMs assessing various aspects of the patients’ experience. These included the widely used generic health-related quality of life measure EQ-5D, and the Oxford Knee or Oxford Hip Score, which were combined with a patient satisfaction question, a global perceived improvement question and a question on surgical complications and hospital re-admissions. Beyond clinical practice evaluation and measuring patient satisfaction, having a centralised and standardised registry has allowed for risk-adjusted comparisons between different surgeons and institutions, and provided further insight into the effectiveness of knee or hip arthroplasty, which are widely delivered and resource-intensive procedures.

'The theory is that we couldn’t improve the quality of care unless we measured it and, at the time, there was no system set up to measure PROMs after hip and knee surgery, so we set up such a system.'

Professor Ian Harris, Steering Committee Chair, ACORN

ACORN involved the voluntary participation of hospitals that perform hip or knee arthroplasty, and, at its peak, 10 NSW hospitals took part. While the registry has now concluded, the standardised collection of PROMs for joint replacements has only expanded. Since 1999, the Australian Orthopaedic Association National Joint Replacement Registry (AOANJRR) has had a long and successful history of collecting and using clinical outcome data (e.g., revision surgery, mortality) from all joint replacement surgeries in Australia, more than 120,000 a year across 315 hospitals. In 2018, the AOANJRR commenced a pilot project in 45 hospitals nationwide, including public and private hospitals, and all those previously involved in ACORN, to collect patient-reported outcomes for hip, knee and shoulder replacement surgeries.

Although run similar to ACORN and using many of the same PROMs, the AOANJRR pilot has primarily collected patient data through a specially-developed, user-friendly web-based platform. This has not only increased the cost-effectiveness of data collection, but enabled direct feedback to all stakeholders through the online dashboard, including patients who receive information on how their experience compares to national averages. The platform also facilitates other research opportunities.

Data collection for the pilot commenced in July 2018 and the 6-month post-operative follow-up will continue up until the fourth quarter of 2019; a formal evaluation will be published at the end of 2019, however, there is already a clear sense of the project’s success and what has contributed to this success. The association of the project with a long running, well-governed registry has strengthened its credibility. Early involvement of surgeons and hospitals was key in getting the PROMs pilot project set up. The ongoing engagement with these stakeholders at site, and particularly the coordinators who identify eligible patients has also been important. Challenges have mainly been system issues including the time required to get multiple ethics and governance approvals, changes in staff acting as site coordinators, identification and recruitment of patient pre-operatively and occasional difficulties in linking patients’ responses. Finally, while there were some initial concerns that web-based data collection would be poorly adopted by the generally older patients receiving joint replacements (average age 60-70 years), in fact response rates have been quite good, with over 90% consenting to pre-operatively, and as many as 80-85% contributing data at 6-month follow up, particularly when a phone call follow up was used. The plan going forward is to secure funding to make the collection of patient-reported outcomes a permanent part of the AOANJRR.

'We’ve developed online dashboard reporting, so patients get their own dashboard and they can see how they compare against national averages... We also give that reporting back to surgeons, to hospitals, to state governments and to project funders. Depending on the level of stakeholder affects what they see in that dashboard… We have had general feedback from people to say it’s very helpful to be able to see national comparison. Currently people use PROMs data to assess a patient on an individual level but they don’t really have an understanding of how that relates to the general public or the national average.'

Grace O’Donohue, Project Manager, AOANJRR

Further information

• ACORN Annual Report 2014

• Australian Orthopaedic Association National Joint Replacement Registry (AOANJRR) website
• EQ-5D Instruments
• Dawson, J., Fitzpatrick, R., Murray, D., & Carr, A. (1998). Questionnaire on the perceptions of patients about total knee replacement. The Journal of bone and joint surgery. British volume, 80(1), 63-69. https://doi.org/10.1016/j.arth.2011.12.015
• Dawson, J., Fitzpatrick, R., Carr, A., & Murray, D. (1996). Questionnaire on the perceptions of patients about total hip replacement. The Journal of bone and joint surgery. British volume, 78(2), 185-190. https://doi.org/10.1302/0301-620X.78B2.0780185

Prioritising patients for elective surgery remains one of the biggest challenges of health systems around the world. To tackle this complex problem, the New Zealand Ministry of Health has been piloting a national surgical prioritisation program to ensure that patients have timely access to elective surgical procedures. A Clinical Prioritisation Tool (CPT) that incorporates clinical, patient centred, and social dimensions is used to identify patients needing urgent access to surgery. Using a national prioritisation approach means that patients who require urgent care have access to it regardless of their age, gender, ethnicity or geographic location, thus, increasing the equity of the health care system. At the heart of the CPT is the locally-developed Impact on Life (IoL) Questionnaire, a patient-reported outcome measure (PROM) designed to capture the needs, abilities and benefits of surgical procedures from the patient’s perspective. Refined over several years, the questionnaire captures six domains: personal safety, personal care, social interaction, personal interaction, leisure activities and the ability to fulfil responsibilities to others. Patients are asked to assess the extent to which their condition affects their ability to perform in each of the domains. The aim is to give the patient a voice by letting them express how badly affected they are by their condition.

The IoL questionnaire is intended for use in making waitlist decisions in all surgical specialties, except high acuity ones such as cardiology and neurology. A variable weighting system allows for the tailoring of the overall contribution of the questionnaire to the CPT score in each specialty, taking into account other clinical aspects such as the potential for clinical deterioration, which may lead to complications and suboptimal patient outcomes. Other factors, such as frequency and duration, are also considered when making the decision about priority for more episodic conditions. Beyond waitlist management, the big advantage of the questionnaire is the capacity to administer it again after a surgical procedure takes place, to understand the benefits patients experienced from surgery.

Practically, administration of this PROM involves emailing or mailing the IoL to patients before their first appointment with a specialist. If not completed ahead of time, there is a backup: the questionnaire can be filled in on-site electronically using a tablet computer. Ideally, data from the questionnaire should be accessible to the clinician at the time they are making the decision about prioritisation. To enable this, the information collected from the patient is available on a web-based prioritisation platform that is accessible by accredited healthcare practitioners.

Recognising the potential for biasing results, patients are encouraged to fill in the questionnaire on their own, that is, with little or no help from their clinician. Indeed, data collected by the Ministry thus far shows that a patient’s assessment of their own condition does not always match their clinician’s severity assessment. The IoL questionnaire transfers some of the assessment power from the clinician to the patient, but this is also a potential barrier to the uptake of the initiative, because it can be perceived as limiting clinical autonomy and decision-making. To overcome this challenge and the risk of clinicians not using the questionnaire, the Ministry is developing extensive health literacy programs to inform patients about their rights and empower them to ask their clinicians about the questionnaire if it is not given to them.

Going forward, New Zealand’s Ministry of Health is working to improve the uptake of the questionnaire before expanding the program to include prioritisation of patients across surgical specialties in addition to within them.

'Patient related outcomes is probably the most important thing we are doing. Our main measure should be how much good did we do for the patient. What I want to know is that this feeds back into the whole thing… PROMs make you ask the question of how much better off this patient is… The more we use PROM the more we get the circle completed and the more we focus on doing the right things and that is more important.'

Mr Christopher McEwan, Clinical Leader, Prioritisation, Electives & National Services, DHB Performance, Support and Infrastructure, Ministry of Health, New Zealand

Further information

• Introduction to the National Clinical Priority System (CPS) for Access to Publicly Funded Hip or Knee Joint Replacement Surgery October 2007
• Chan, G., Bezuidenhout, L., Walker, L., & Rowan, R. (2016). The Impact on Life questionnaire: validation for elective surgery prioritisation in New Zealand prioritisation criteria in orthopaedic surgery. The New Zealand Medical Journal (Online), 129(1432), 26.