Guiding recommendation: Ensure systems are in place for rapid and transparent evidence collection, synthesis and evaluation.
The operating environment during the alert phase of a pandemic involves high levels of uncertainty, and there is an imperative for governments to act without the level of evidence that would normally apply. The more evidence that is available to decision‑makers, the more confident they can be in their actions. Achieving the best possible outcomes requires interoperable data systems, investment in the capacity to collect and synthesise evidence, and a commitment to ongoing evaluation and evidence‑based decision‑making to ensure proportionality is achieved and maintained.
At the start of the pandemic there was limited knowledge about the origin of the virus, the method of transmission, the mechanism by which the virus caused illness and death, who was most at risk of severe disease, and the most effective treatment options. It was also unclear whether it would be possible to develop a vaccine. This required operating under the ‘better safe than sorry’ precautionary principle, which does not necessarily require evidence to act but prioritises protecting lives.
Moving to an evidence‑based risk approach requires real‑time evidence relevant to the Australian context. In the early stages of the pandemic, a lack of sufficiently detailed Australian data meant that epidemiologists, modellers and other experts relied on international data that did not necessarily reflect the risk profile or disease dynamics in Australia. The more Australia’s experience of COVID‑19 deviated from that of the rest of the world, particularly from Europe and the UK, where the data we heavily relied upon originated, the bigger the issue became. The accessibility and security of data was also an issue, with case and close contact data being recorded in Australia in hard copy and relayed to recipients by fax machines in some jurisdictions.
Data availability improved through the pandemic as Australian Government departments collaborated with each other, and with state and territory governments, to create, share and link datasets, building a more complete picture of the pandemic. This required overcoming entrenched barriers to data‑sharing and innovation to create new data systems. The evidence base this created was invaluable to decision‑makers.
Particular successes included:
- the Department of Health’s rapid integration of multiple datasets held across different Australian Government departments, different jurisdictions and providers in order to increase understanding of vaccine uptake, drive research on the effectiveness of the COVID‑19 vaccine and inform other tailored response measures, including for priority populations
- the Treasury obtaining credit card spending data from banks and mobility data from private companies like Google to monitor the effectiveness of lockdowns in reducing activity, and the economic impact of restrictions and response measures
- the Australian Bureau of Statistics (ABS) linking Australian Taxation Office, Treasury and ABS Multi‑Agency Data Integration Project and Census data so that the Treasury could understand how coming off JobKeeper affected individuals
- the use of wastewater testing to detect COVID‑19 at the local population level or on aircraft. The results signalled where public officials should target public health messaging and helped identify asymptomatic cases and new variants on incoming flights.22
However, gaps remained, particularly regarding the impact on different cohorts of Australians and the greater investment needed in interoperable data systems to provide the evidence to underpin a more effective, tailored, evaluated and equitable pandemic response. To this day, COVID hospitalisation counts – an important measure of impact – do not mean the same thing in different jurisdictions and so cannot be directly compared.
Inadequacies in disability data impacted governments’ ability to develop tailored response measures, understand the extent of cases and deaths, or monitor the rollout of vaccinations. This is concerning given that some people with disability have a greater risk of acquiring COVID‑19 and are more likely to have serious health consequences as a result. During the Delta wave, people receiving the Disability Support Pension and National Disability Insurance Scheme participants were 3.1 and 2.8 times more likely than the general population to be admitted to intensive care units with COVID‑19. These rates increased to 4.7 and 4.8 times respectively in the first Omicron wave. While data linkages were created to track the vaccine rollout amongst some people with a disability, there is no longer a clear picture on vaccine coverage among this group. Regular boosters remain an important protective measure for people vulnerable to more severe COVID‑19, and understanding where uptake is dwindling would help prioritise public health community engagement efforts.
Evaluation is a critical component of ensuring that government pandemic response measures are effective and remain appropriate in changing conditions. The Australian National Audit Office continued to undertake audits throughout the pandemic but adjusted their approach to account for changing circumstances. This helped ensure accountability while not losing the value of the audit function.
Public health interventions similarly required – but often lacked – real‑time evaluation on implementation. A number of the more contentious measures, such as enforced quarantine, curfews and closing outdoor playgrounds, were not supported by pre‑existing evidence, and would have benefited from being tested with epidemiologists and behavioural experts, and evaluated in real time to ensure proportionality.
There were also a number of economic response measures that had not previously been used in Australia. In particular, the pandemic saw the first use of a wage subsidy in the form of the JobKeeper payment. This represented the single biggest program by annual government spending in Australia’s history, and the three‑month review of the program allowed changes that improved its operation during its second phase.
A post-pandemic evaluation of JobKeeper was also undertaken, strengthening future preparedness and ensuring detailed lessons have been learned. However, outside JobKeeper and the Reserve Bank of Australia’s reviews of their extraordinary monetary policy measures, a range of other significant economic programs – including the $35.9 billion Boosting Cash Flow for Employers program – have not been formally evaluated. Reviewing the full range of response measures deployed would maximise and consolidate the learnings for a future pandemic.
Lessons for a future pandemic
Pandemics invariably involve making decisions in the face of significant uncertainty. However, the existence of strong, secure, readily adapted, interoperable data systems, processes and capacity for generating, synthesising and communicating evidence can reduce this uncertainty by providing governments with the evidence‑based intelligence they need to assess risks and minimise harm.
In order for this to occur, pre‑agreements to collect, link and share data across the Australian and state and territory governments need to be established and maintained, allowing the real‑time collection and use of relevant data at the start of a pandemic. If existing systems are found to be insufficient for supporting decision‑makers early in the next crisis, there need to be mechanisms in place to identify this and rapidly establish new data collection and sharing arrangements.
Collecting specific health and socio‑demographic data on priority populations needs to be part of the comprehensive integrated data system to inform responses tailored to their unique circumstances and needs. National research capability also needs to be harnessed as part of the national real‑time evidence asset and coordinated nationally.
Ongoing real‑time evaluation is required. Such evaluation should allow for the monitoring of impacts as conditions and risk change, community adherence shifts, new variants of the infectious agent emerge, more effective treatments are discovered and immunity is acquired.
Evaluation should also encompass broader health impacts that may be unintended consequences of public health measures, including differences in public health measure effectiveness or the severity of negative collateral impacts across different population groups.
Governments should also implement approaches for evaluating non‑health response measures during a pandemic. In a crisis, governments should create ‘feedback loops’ to assess the efficacy of measures and to adjust or remove them in line with findings. In doing so, governments would ensure that measures remain appropriate, and are only employed for the time when benefits outweigh costs.
Comprehensive post‑action reviews for all major response measures should be undertaken to determine the successes and lessons to be learned. The findings from reviews should inform responses to future pandemics. To ensure the success of this process, all government departments should take responsibility for leading the evaluation of their own response measures.
Back to topImmediate actions
The Inquiry recommends the following immediate action to strengthen evidence and evaluation approaches over the next 12 to 18 months:
- Improve data collection, sharing, linkage, and analytic capability to enable an effective, targeted and proportionate response in a national health emergency, including:
- improvements to timeliness and consistency of data collection and pre‑established data linkage platforms across jurisdictions, including for priority populations
- expanded capability in Australian Government departments to gather, analyse and synthesise integrated economic, health and social data to inform decisions
- finalising work underway to establish clear guardrails for managing data security and privacy and enabling routine access to linked and granular health data, and establishing pre‑agreements and processes for the sharing of health, economic, social and other critical data for a public health emergency to ensure rapid mobilisation of real‑time evidence gathering and evaluation.
Medium‑term actions
Ahead of the next pandemic, the Inquiry has identified the following medium‑term action for government:
- Continue to invest in monitoring and evaluating the long‑term impacts of COVID-19, including long COVID and vaccination adverse events, mental health, particularly in children and young people, and educational outcomes.