When AI Gets It Wrong: The Leadership Work of Recovery, Trust, and Learning

Most leaders are preparing for AI adoption. Fewer are preparing for AI recovery.

That difference plays an important role. Because AI will not only make service faster, more personalised, and more efficient. It will also create new moments where customers, staff, suppliers, or communities feel confused, exposed, misled, or poorly served.

A chatbot may give the wrong advice.An automated recommendation may be technically plausible but commercially unfair.A staff member may trust an AI-generated answer too quickly.A customer may be passed between digital and human channels without anyone taking real ownership.A sustainability claim may be repeated before the organisation can fully evidence it.

Not every mistake will be dramatic. Many will look small at first: a poor handover, a vague response, a decision no one can explain, a customer who feels the system has stopped listening.

But each moment teaches people something important about the organisation.

Can it notice harm early?

Can it tell the truth without becoming defensive?

Can it learn without blaming the tool, the team, or the customer?

Can it slow down enough to rebuild trust before scaling further?

That is the leadership work of AI recovery.

New Zealand’s first AI Strategy, released in July 2025, is a useful trigger for this discussion because it encourages private-sector AI adoption while also anchoring that adoption in responsible use. The strategy says New Zealand is deliberately focusing on AI adoption and application rather than foundational AI development. It also states that New Zealand is a signatory to the OECD AI Principles, which it describes as the foundational international framework for AI governance.

Those OECD principles include human rights and democratic values, transparency and explainability, robustness, security and safety, and accountability.

MBIE’s companion Responsible AI Guidance for Businesses makes the leadership implication more practical. It identifies governance and accountability, supporting capabilities, stakeholder interactions, data and modelling, and use of AI outputs as core areas for responsible AI use. It also states plainly that AI tools can be helpful but do not always get things right, and that AI should support people’s work rather than replace human judgement.

That is a practical message for leaders: responsibility does not disappear when a task becomes automated.

AI will create service failures as well as service gains

Much of the AI conversation still begins with efficiency.

How much time can we save?

How quickly can we respond?

How much personalisation can we offer?

How many routine tasks can we automate?

These are valid questions. But they are falling short in many respects, because in real service environments, customers do not experience AI as an abstract technology. They experience it through moments: a reply, a recommendation, a form, a claim, a decision, a delay, a refund, a handover, a conversation.

When those moments work well, AI may feel invisible.

When they go wrong, the customer often does not separate the tool from the organisation.

The organisation said this.

The organisation recommended this.

The organisation made this difficult to challenge.

The organisation made me feel unseen.

That is why AI-enabled service failure is not only a technical incident. It is also a trust event.

The OECD’s work on AI risks and incidents supports this wider view. It notes that AI risks need to be managed throughout the AI value chain, that those who deploy AI must be accountable for the proper functioning of their systems, and that monitoring incidents and hazards helps build the evidence needed to understand where AI risks materialise.

For leaders, the implication is clear: responsible AI cannot be judged only by launch quality. It must also be judged by the organisation’s capacity to detect, explain, correct, and learn from failure.

Recovery is a leadership capability

When AI-enabled service goes wrong, the first instinct is often technical.

Fix the model.

Adjust the prompt.

Change the workflow.

Update the data.

Add another approval step.

Those actions may be necessary. But recovery requires more than a technical patch.

Leaders must decide what gets escalated.

They must decide who speaks to the customer.

They must decide whether staff are allowed to question the system.

They must decide whether the organisation names the issue clearly or hides behind vague language.

They must decide whether speed is still the priority, or whether the moment now requires care.

In other words, recovery is a leadership capability.

MBIE’s guidance supports this because it links responsible AI use to oversight, clear accountability, stakeholder communication, risk management, documentation, testing, monitoring, and staff capability. It also says businesses may need to know where and how AI is being used in order to respond to customer queries or audits, and that good documentation should be followed throughout the AI lifecycle.

A strong recovery response usually has several parts.

First, the customer’s experience is taken seriously, not minimised.

Second, ownership is clear.

Third, the organisation explains what it knows, what it does not yet know, and what it is doing next.

Fourth, staff are protected from blame long enough to understand the system properly.Fifth, learning is captured before the organisation rushes back to business as usual.

This is where many organisations struggle. Not because they lack intelligence or intent, but because failure creates pressure. Pressure narrows attention. It makes people defensive. It can pull leaders into technical language when the customer needs plain language. It can encourage teams to protect the project rather than examine the promise.

The recovery moment asks leaders to do something harder: stay steady enough to learn.

The coaching challenge underneath

AI recovery is often discussed in governance language, and rightly so. Organisations need clear responsibilities, evidence, documentation, escalation paths, and human oversight.

But underneath the governance work exists a human challenge.

When something goes wrong, leaders may feel exposed. Teams may worry about blame. Staff may become reluctant to use judgement because the system appears more authoritative than they feel. Customers may arrive already frustrated because they have had to fight through a process that did not recognise their situation.

This is where leadership behaviour comes in.

Can the leader ask better questions before giving answers?

Can they stay curious when the first explanation is uncomfortable?

Can they separate accountability from blame?

Can they listen to front-line staff who saw the failure before the dashboard did?

Can they hold the commercial, ethical, operational, and human dimensions together?

Coaching is relevant here because the failure moment is not only about what leaders know. It is about how they show up under pressure.

A reactive leader may defend the system.

An anxious leader may over-control the process.

A vague leader may reassure too quickly.

A steady leader can create the conditions for truth, repair, and learning.

That steadiness is not soft. It is operationally useful.

It helps customers feel heard.

It helps staff speak honestly.

It helps the organisation find the real weakness rather than the most convenient explanation.

Five questions leaders should ask after an AI-enabled service failure

After an AI-enabled service failure, the best question is not simply: “What went wrong with the tool?”

The better question is: “What did this failure reveal about the system we have built around the tool?”

Here are five questions worth asking.

1. What happened from the customer’s point of view?

Internal explanations often begin with process. Customers begin with experience.

What did they expect?

Where did they feel confused, ignored, misled, or trapped?

At what point did the service stop feeling human enough?

This question is so vital because trust is rebuilt from the outside in.

MBIE’s guidance says clear communication with impacted stakeholders, including staff, customers, clients, shareholders and other impacted parties, is important for building and maintaining trust and confidence in how AI is used and developed. It also notes that stakeholder engagement can help organisations identify and respond to problems early.

2. Where did human judgement leave the process too early?

Many AI failures are not caused by automation alone. They happen because the organisation quietly removes human judgement from the moments where it is still needed.

Who checked the answer?

Who had permission to override it?

Who noticed the exception?

Who was accountable for the final decision?

Human oversight is not a decorative safeguard. It must be real, skilled, and usable.

This is directly supported by MBIE’s Responsible AI Guidance, which says AI should support work rather than replace human judgement, with people staying responsible for final choices. The AI Strategy also identifies maintaining human oversight of critical decisions as an ethical consideration.

3. What did the system optimise for?

AI-enabled systems often optimise for speed, consistency, scale, cost, or conversion.

But what did that optimisation push into the background?

Fairness?

Context?

Customer vulnerability?

Staff discretion?

Long-term trust?

A system can perform exactly as designed and still create the wrong outcome. This is where the OECD AI Principles are useful. They ask AI actors to consider human rights and democratic values, including fairness and privacy, as well as transparency, safety, security and accountability across the AI system lifecycle.

MBIE’s AI Strategy states that New Zealand is a signatory to the OECD AI Principles, which it describes as the foundational international framework for AI governance.

4. Who carried the risk when the mistake occurred?

This may be the most important leadership question.

Did the customer carry the inconvenience?

Did front-line staff carry the emotional labour?

Did a supplier carry the reputational risk?

Did a vulnerable group carry more of the downside than others?

Did the organisation benefit from automation while someone else absorbed the failure?

Responsible AI requires leaders to notice not only whether value is created, but how risks and benefits are distributed.

The OECD’s work on AI risks and incidents supports this broader framing. It identifies harms such as bias and discrimination, privacy infringements, safety and security issues, and workplace risks including privacy, work intensity, bias, discrimination and accountability.

5. What must change before we scale this further?

The temptation after a failure is to fix the visible issue and keep moving.

But the better leadership move is to pause long enough to learn.

What needs to change in training?

What needs to change in escalation?

What needs to change in customer communication?

What needs to change in data, governance, supplier agreements, or staff authority?

What evidence would show that the organisation is ready to scale responsibly?

Scaling without learning simply spreads the weakness faster.

This is supported by MBIE’s guidance on continuous monitoring and improvement, lifecycle documentation, risk management, AI inventories, recordkeeping, testing results, identified risks and mitigations, stakeholder identification, and mechanisms for human control and oversight. It is also supported by the OECD principle that mechanisms should be in place, as appropriate, so that AI systems can be overridden, repaired, or decommissioned safely if they risk undue harm or show undesired behaviour.

The real trust test

The real test of responsible AI is not whether an organisation can launch a tool.

It is whether the organisation can remain accountable when the tool produces an outcome people did not expect.

That requires more than technical competence. It requires leadership discipline, customer empathy, honest communication, staff confidence, and learning loops that are strong enough to change the system.

AI adoption will continue. So will digital service, data-driven decision-making, automated support, and AI-augmented sales and service. The organisations that earn trust will not be the ones that promise perfection. They will be the ones that recover well, learn visibly, and improve before the next failure becomes larger.

Responsible AI is not proven by the launch.

It is proven by what leaders do when the first serious mistake happens.

Start the conversation