"AI cannot make human-centred decisions and cannot understand wants or be empathetic" (Workshop participant)
Emerging technologies such as AI may fundamentally shape not just how public services are designed, but how people interact with government. While this could lead to significant improvements in service performance (as outlined above) there is a view that it could come at the expense of empathy for individuals.
I would expect cost savings and time reductions but a depersonalising of interactions and a bureaucratic adherence to process.
Presently, AI is not well suited to provide tailored customer service to the public and will lack the nuance and empathy that the public are likely to expect when they try to engage with a service.
AI-enabled public service delivery offers opportunities to improve performance by streamlining access to services for many in the community. While people are comfortable with using AI to improve process and operational efficiency, when it comes to applications where a decision maker has a lot of discretion in decision-making, some workshop participants and survey respondents questioned whether human complexity could be translated into an AI system.
So in some ways I’m happy enough to sit by and do my tax online and have the little question box…So for a pretty simple response that's yes/no/go here, that's all fine. Of course it's helpful. But when you actually get into parts of the human condition that are intrinsically based in behaviour and complex behaviours…how do you actually translate that into an AI space?
I highly value a great user experience through digital channels including on a self-service basis. However, I still also value the ability to talk to someone to escalate an issue or obstacle including delays or inability to find what I am looking for via digital channels…
A loss of human interaction in moments of need could significantly erode trust. Research literature indicates that a lack of interpersonal interaction with public service actors and decision-makers is a significant driver of distrust in AI, while community representatives noted that AI might create barriers to getting in touch with a real person in moments of need (Future Scenario 2).
Future scenario 2: Climate’s influence on the system
In one scenario, workshop participants were asked to consider a future where an increase in climate driven events required a redistribution of public resources. Most routine and transactional services were automated to allow for greater focus on rapid support for people impacted by crisis such as floods, bushfires and cyclones. Human-to-human services were largely reserved for high-trauma and high-impact incidents. To receive automated services people needed to identify themselves using their biodata, those refusing to provide biodata would need to apply for and wait for in-person services to become available. This scenario helped us to understand:
- The willingness of people to access the majority of public services via AI-enabled digital platforms without engaging humans as part of the process, and gather insights on whether people are willing to trade potentially slower human-based service delivery for potentially faster AI-based service delivery.
- The willingness of participants to use biometric data as a means for personal identification, and gain a response to a proposition that those who do not wish to share biometric data will not receive as prompt a service as those who share biometric data as a means for identification.
We learnt that people:
- Understand the need to prioritise a response to those in crisis, but do not think that people who preferred to interact in person should receive a slower service.
- Are concerned about the level of service provided after a crisis and how local communities would be resourced for the longer term work of rebuilding resilience in a community after an adverse event.
- Are very concerned about the use of personal data (and biometric data specifically), even in a crisis; and the impacts on those who would opt out of sharing their personal data altogether.
- Feel sympathy for those who might want to opt out of sharing personal data because of their experiences and concerns that their past history might discriminate against them.
- See a risk of a greater social divide for those who are experiencing complexity and vulnerability, and need ongoing public services through human engagement.
Trustworthiness is built when the public service demonstrates concern and empathy for the people it serves
Traditionally, public service delivery has included face-to-face interactions, which provide an opportunity for public servants to recognise and respond to an individual’s circumstances with empathy in the moment. Community representatives and experts in workshops highlighted the importance of those interactions, noting that many people desire to talk to, be seen by, and receive help from a real person who can empathise and offer a service suited to their unique circumstances.
In order to promote trustworthy public services, the APS must ensure that its services are human-centred and responsive to human need and the nuance of diverse experiences. This includes by increasing accessibility, clarity, responsiveness, and transparency. It also requires clear and tangible demonstrations of ethical decision-making, fairness and commitment to the public interest.
More generally, respondents to the Have Your Say survey highlighted the importance for trustworthiness of a public service that is seen to act in the best interests of Australians:
Trust means people are putting their faith in the public service to do right by them.
It is likely that in many instances, AI-augmented processes could help frontline staff to engage in stronger and more in-depth relationships with users than is currently the case. Again, workshop participants highlighted that design input from those who use public services could help ensure that AI-enabled services remain human centred, and even help agencies to become more humane and compassionate:
Culturally and trauma-informed AI doesn’t exist, but if we could get there in the future, that could be really special and make it easier” [for agencies and for the people that engage with them].
On the other hand, it was felt that AI is not capable of empathy. While AI’s human-like capability can compel users to perceive an AI-system as a person,[46] this was seen to pose a significant risk to trustworthiness, with focus group participants noting that fake empathy from AI could completely destroy trust. Equally, there is a risk that decisions and outcomes that are seen to be lacking empathy are attributed to the use of AI – whether or not this is justified – reducing trust in government’s ability to use AI responsibly in service delivery.
Insight 2.1: People want enough of a relationship with public services – what that looks like depends on an agency’s trust history, the community it serves, and the type of service it offers
For some in the community, human interaction and a relationship with public services is as important as the service itself for demonstrating trustworthiness. For many people, interacting with services can be confusing or overwhelming. Direct contact with a person can reduce some of that discomfort and make it easier for them to access and engage with public services. In contrast, other groups may not want or need to access public services in person — for some people, convenience may be much more important:
[Potential benefits of AI use in public service] “Shorter wait times, access to services when you need them (24/7, 7 days a week)…
Considering the community as a whole, what matters is ensuring enough of a relationship. For a given agency, this will depend on several factors:
An agency’s trust history…
Community representatives noted that how communities have been treated in the past and in times of crisis will determine future interactions and perceptions of trustworthiness. For example, responses to the Have Your Say survey highlighted how the Robodebt Scheme had undermined trust in public service agencies:
Robodebt killed much of my trust in public services. Trust is gradually being redeemed through compensation and action, but it's too little, too late, for too many.
…the community an agency serves…
Community representatives emphasised that the attributes and actions that are seen to build an agency’s trustworthiness are different for different cohorts of the community, and that cultural differences come into play. In particular, empathy dimensions of trustworthiness are likely to be particularly important for those experiencing greater vulnerabilities and those with more complex needs. Participants in workshops also emphasised the importance of person to person relationships – which participants suggested are the opposite of AI – for building trustworthiness with First Nations peoples, noting that First Nations peoples want to be deeply listened to and heard by governments.
All of the things we are talking about with AI/tech developments are the complete opposite of what First Nations communities want. We know they want relationships, they want to be listened to deeply, they want to spend time with government, and have people live among their communities to understand their perspective and way of life.
…and the type of service an agency offers
Participants in workshops suggested that there were opportunities to use AI for transaction type interactions with the public service, as people just want to get what they need, making empathy much less important.