User Guide for Reviewing Clinical Variation

Select areas of clinical care to investigate based on:

• High volume of patients
• High risk to patients, regardless of volume
• High morbidity, mortality or patient dissatisfaction
• Clinical areas identified in the health service’s risk register as high or potentially high risk
• Existence of an established evidence base for best practice
• Evidence that overuse or underuse of the intervention increases risk to patient health
• Advice from clinical leaders in the health service organisation
• Availability of external data or standards/guidelines for comparison with the health service organisation’s practice
• Clinical areas where new evidence or technology has substantially changed the standard of care
• Interventions that have been identified as low-value care
• Choosing Wisely Australia recommendations.

Decision making about selecting the priority areas to investigate can be aligned with the

organisation’s risk management approach (Action 1.10 of Standard 1, Clinical Governance).

Once you have chosen the clinical area, you will need to decide whether you will compare your data against data from other health service organisations, and/or evidence-based guideline recommendations or a clinical care standard. This decision will be influenced by the availability of these comparators.

You will also need to decide whether you will use clinical team data, departmental-level data and/or health service organisation-level data, for comparison with external data, guideline recommendations or standards.

In some cases there may be substantial variation within the health service organisation even if the data does not show marked variation at a whole-of-organisation level compared with external data or with guidelines. For this reason, exploring the data at the level of clinical teams as well as whole-of-organisation level is very useful for ensuring appropriate care.

This step requires health service organisations to have processes that:

• Measure clinical care delivered and outcomes achieved in the health service organisation
• Compare the care delivered or outcomes achieved within the health service organisation to those of other organisations, using external reports, clinical quality registries and audits, and/or
• Assess the extent to which care delivery and outcomes align with evidence-based guideline recommendations and clinical care standards
• Assess the clinical importance of any noted variation and take action if required. 

Determine the measures or indicators 

Use a clear, consistent definition of the measures or indicators to be assessed to allow meaningful comparison with other health service organisations’ data and with guidelines or standards. In many instances there are existing high-quality indicators that can be used to consistently measure data over periods of time, and that allow comparison with results from similar health service organisations. For example, Clinical Care Standards include indicators that can be used.

Define the target population

Other key requirements include a definition of the target population – the group of patients who should be offered the specific type of care outlined in the clinical care standards or guideline recommendations – and identification of the benchmarks that should be achieved within the health service organisation.

Audit the clinical records

Clinical records can then be audited using the systems established for this purpose. Action 1.16 of the NSQHS Standards requires the health service organisation to have healthcare record systems that support systematic audit of clinical information. If electronic systems are used to extract information it should be possible to determine whether care was provided to all eligible patients within a certain time period. However, if manual collection of information from medical records is required, it may only be feasible to collect information from a sample of patients. Once the information has been collected it can be summarised, and the extent of any identified gaps between best evidence and current practice can be assessed. See Using data for quality improvement.

Auditing the relevant aspects of patient records is an effective way to gain insights into how and why clinical care is deviating from best practice. Ensuring that relevant staff discuss the findings as a group will draw out different perspectives on processes that affect clinical care, and can help foster a team approach to improving practice.

Reasons for clinical variation

There are many reasons for variation in rates of care processes and in outcomes of care. Variation can be due to differences in the needs of patients, in which case it is warranted and desirable. Other reasons for clinical variation include:

• Clinical care not changing in line with updated evidence
• Differences in clinicians’ knowledge of the latest evidence or skills related to new diagnostic or interventional procedures
• Clinical uncertainty about an intervention’s place in therapy, and the need for better data on its benefits and harms
• Inequity of access to care
• Effects of financial incentives or disincentives
• Inadequate system supports for appropriate care
• Inadequate information sharing and discussion with consumers
• Barriers experienced by Aboriginal and Torres Strait Islander peoples in accessing appropriate, culturally safe care.

It is important to give patients clear information about options, and to provide adequate opportunity to discuss this information. The Commission has a range of resources to assist with shared decision making so patients can make properly informed choices. For some interventions there is a lack of data on benefits and risks; in this situation it is important to discuss what is known and what is unknown, and what the other options are.

Unwarranted variation

Relatively low or high rates of use of an intervention may arise for reasons other than decisions made by individual clinicians. For example, low use of some interventions may signal a problem with access to clinical care for people who need certain tests, treatments or procedures. In this instance the health service organisation may need to examine whether services, and the workforce and resources to deliver them, are appropriately allocated given the needs of people within their catchment area. However, unwarranted variation in use may also indicate ongoing use of an out-dated method of treatment, and signal the need for clinical training in new procedures or treatments. Unwarranted high rates of interventions may also occur because of an oversupply of workforce or technology leading to lowering of clinical thresholds for undertaking these interventions.

Occasionally the investigation will highlight a pattern of problems with individual or team decision-making or skills. In this situation: consider what extra support is needed to deliver the best possible care; determine whether an external review is needed with the relevant manager; and follow the performance review processes in accordance with Action 1.22 of the NSQHS Standards. Also, implement the incident management systems (Action 1.11 of the NSQHS Standards) and open disclosure (Action 1.12 of the NSQHS Standards) when relevant.

For detailed advice on auditing patient records and other methods of investigating reasons for clinical variation, see the resources listed under Using data for quality improvement in Step 3.

Actions to improve appropriateness of care that are prompted by your variation findings should be incorporated into the organisation’s overall approach to safety and quality improvement.

Continuous quality improvement occurs through a cyclical approach:

1. Specify the desired goal for improvement
2. Explore the reasons for current practice
3. Identify the barriers or enablers for any desired change in practice
4. Make changes to health care processes
5. Monitor progress and make further changes as necessary.

Actions to improve appropriateness of care will need to be embedded in the health service organisation for improvements to care to be sustained. See Using data for quality improvement for resources offering detailed advice on data-driven quality improvement methods.

‘Case for improvement’ documents

‘Case for improvement’ documents, which are available for some of the clinical care standards, provide further useful information for quality improvement. For each quality statement in a clinical care standard, the case for improvement document asks the following questions:

• Why is it important?
• What is known about current practice?
• What could be achieved with more consistent application of the aspects of care described?

When possible, examples are provided showing how specific approaches or systems for implementing best practice have demonstrated measurable change.

​​Do not delay investigating patient care to check data

If you have found that practice within the health service varies substantially from other health service organisations or from evidence-based guideline recommendations or clinical care standards, the priority – and prime responsibility – is to ensure there is no problem with patient care. Do not delay investigating potentially suboptimal care in order to recheck data.

The health service organisation should maintain records of reviews and actions taken as a result of examining variation and appropriateness of care. Processes should be in place to report actions and outcomes, consistent with the requirements of Action 1.09 of the NSQHS Standards.

Clinicians have a professional responsibility to review the care they provide and to participate in efforts to improve the quality of care. Organisations employing clinicians need to ensure that there is clinical participation in internal reviews of clinical care performance, and should encourage and provide practical support for clinical participation in relevant external clinical quality registries and audits.

Health service organisations should keep records of clinician participation in reviews of clinical variation.

Clinicians can use these records to demonstrate compliance with continuing professional development requirements; for example, the continuing professional requirements for medical practitioners. This aligns with Action 1.22 of the NSQHS Standards.

Requirements for continuing professional development

Stronger continuing professional development is one of the core features of the Medical Board of Australia’s new Professional Performance Framework for medical practitioners which comes into effect from January 2023. As well as participating in educational activities to build knowledge and skills, medical practitioners will be required to review their performance and measure the outcomes of care they provide.

The expectation that health service organisations review variation and appropriateness of clinical care is consistent with the expectation of the registering body that individual medical practitioners should regularly review their performance and outcomes.