Minister increases pressure for better evaluation

The event, titled Impact evaluation: assessing the effectiveness of Australian public policy, featured the cream of Australia’s evaluators and a keynote by French American economist Esther Duflo, who won a Nobel prize for her groundbreaking use of randomised controlled trials (RCTs) to alleviate poverty.

Professor Duflo – speaking from Paris – joined a string of leading evaluators at the event, and her talk highlighted the many applications for RCTs, as well as their ability to influence public policy.

Other speakers included:

• Dr Jenny Donovan from the Australian Education Research Organisation
• Professor Sharon Goldfeld from the Murdoch Children’s Research Institute
• Professor Julian Elliott from the Future Evidence Foundation
• Dr Robyn Mildon from the Centre for Evidence and Implementation
• Dr Alex Gyani from the Behavioural Insights Team.

As previously reported by Grants Management Intelligence, Dr Leigh has strongly urged those working for the public service to adopt RCTs in assessing the effectiveness of their work.

RCTs are considered the “gold standard” for evaluation, but many practitioners have doubts about whether they are the best method for evaluating public policy efforts because of their expense. Dr Leigh cited a string of case studies that demonstrated their effectiveness.

“Through the Australian Centre for Evaluation, we are keen to bust the myth that high-quality evaluation is expensive, complicated and time consuming. Some randomised trials can be pricey and continue for decades. But the four studies I cited … were cheap and cheerful, taking less than a year, and costing thousands of dollars, not millions.”

He urged public servants and practitioners to challenge “self-evident truths”, in which false assumptions about education, health, First Nations priorities and many other government activities, meant that evaluations were often “an afterthought rather than built into policy design”.

Dr Leigh also said just 1.5% of evaluations in educational policy and other government programs used randomised trials, but he believed this would change as policy makers adopted a “what works?” approach.

“Just as evidence has increasingly been institutionalised into the training of medical professionals, it is likely that evaluation will become part of the standard toolkit for all Australian public servants. I like to imagine that as public servants apply for promotion to the senior executive service, they will be asked not only ‘tell me about a management challenge you addressed’, but also ‘tell me about the best evaluation you ran.'”