Platform, Partner or Network Layer? How Credit Unions Should Evaluate Conversational AI

Put a credit union's manager, IT lead and board risk officer in a room to discuss conversational AI, and each will mean something different by "the platform". The manager means the thing members talk to. The IT lead means the system it runs on. The risk officer means the company that's accountable when it goes wrong. All three are right, and that's why evaluations stall: they're three decisions being treated as one.

Credit unions should evaluate conversational AI as three separate decisions: the AI platform that understands and answers members, the implementation partner that integrates and runs it, and the network layer that carries the calls. The best result comes from judging each on its own criteria, then checking that one party is clearly accountable for the whole service.

TL;DR

• "Conversational AI platform" usually hides three choices: the AI, the partner and the network.
• The AI platform is judged on understanding, accuracy and channel coverage.
• The partner is judged on integration, handoff design, support and regulatory evidence.
• The network layer is judged on availability, voice quality and whether AI can run where the calls already travel.
• A pilot scored on member outcomes beats any demo.

What are the three decisions?

The AI platform is the software that understands what a member says or types, works out what they want, and responds. It might sit inside a contact centre platform, come from a standalone vendor, or run in the voice network.

The implementation partner connects the AI to core banking, designs the call flows and handoffs, supplies the evidence a regulated credit union needs, and supports it after launch.

The network layer carries the calls. For phone-based AI, it decides availability and audio quality, and whether the AI can sit in the call path without the credit union changing its phone system.

Sometimes one company covers all three. Often it doesn't. Either way, each needs judging on its own terms. DigitalWell's earlier guide to choosing conversational AI for Irish financial services compares the main deployment models; this article focuses on how to evaluate each layer.

How should a credit union judge the AI platform?

Judge it on what members experience. Four tests matter most.

• Understanding. Does it handle Irish accents, members who don't use the "right" words, and requests that change halfway through?
• Accuracy. Does it give the correct balance, loan status or branch hours every time, from live data?
• Disclosure. Does it tell members they're speaking to an automated assistant, as Article 50 of the EU AI Act has required since 2 August 2026?
• Channels. Does it cover phone as well as chat? For many credit unions, the phone carries the most member contact.

Test these with real member scenarios written in your members' own words.

How should a credit union judge the implementation partner?

Judge the partner on what happens around the AI. This is where much of the effort and risk sits.

• Integration. Can it connect to your core banking system for both reading and, where needed, writing data?
• Handoff design. When a member needs a person, does the staff member get the verified identity and the conversation so far?
• Regulatory evidence. Can it supply what section 76J of the Credit Union Act 1997 and the Central Bank's outsourcing and operational resilience guidance call for: a written agreement, performance reporting, incident logs, continuity testing and exit terms?
• Support after launch. Who tunes the AI, fixes faults and adds new call types?

DigitalWell's Credit Unions page sets out the supplier documentation it provides: corporate and data-location statements, security posture, SLA and KPI performance, incident logs, proven access for auditors and the Central Bank, and joint continuity testing.

How should a credit union judge the network layer?

Judge it on availability and position. If the AI answers the phones, the network decides whether members can reach it.

Ask for the provider's uptime history on the network your calls will use. Ask whether calls can be rerouted to another branch or team if a site goes down. And ask whether the AI runs inside the network, or whether calls have to be sent out to another provider's cloud and back.

Martin Browne, CTO and CX Chief Technologist at DigitalWell, describes DigitalWell's approach:

"It's our system running on our network, our voice network."

Martin Browne, CTO and CX Chief Technologist, DigitalWell

DigitalWell's AI Voice Network runs AI Voice Agents inside its own licensed carrier network, which carries over 100 million minutes of voice traffic. Because the AI sits where the calls already travel, it works with the credit union's existing phone system, and every call is transcribed, summarised and logged. DigitalWell's article on what resilience should mean when a credit union puts AI on the phones covers failure points and fallbacks.

Who is accountable for what?

The test of a good arrangement is that every part of the service has a named owner. The table shows a typical split, and where gaps appear when the three layers come from different companies.

Where one provider covers the partner and network roles, as DigitalWell does for its AI Voice Network, most of the gaps in the right-hand column close. The credit union still needs to check who owns the AI model, where data is processed, and how the exit would work.

How should a credit union score a pilot?

Run a pilot on one or two call types, then score it on member and operational outcomes. Agree the measures before the pilot starts.

1. Share of calls completed end to end by the AI, with no staff involvement.
2. Accuracy, checked by reviewing a sample of calls against core banking.
3. Handoff quality: whether staff received the context and whether members repeated themselves.
4. Member feedback from a short post-call question or a sample of follow-up calls.
5. Availability during the pilot, including any failover events.
6. Staff time released, measured against the same weeks before the pilot.
7. Evidence pack: whether the provider delivered the reporting your board and outsourcing review need.

A pilot that scores well on all seven is a stronger case for the board than any demo.

Frequently Asked Questions

Do we need to buy the AI platform, partner and network from one company?

No, but fewer parties make accountability simpler. Where they're separate, map who owns each responsibility and write it into the contracts.

Which matters most: the AI, the partner or the network?

Often the partner. Integration, handoff design and regulatory evidence decide whether the AI works in practice.

Can we use the AI built into our contact centre platform?

Yes, if you're already on a platform with built-in AI. Check it covers your phone volume and that the partner can integrate it with core banking.

Does DORA affect this decision?

Not yet. Credit unions are exempt from DORA until 17 January 2028. Section 76J and the Central Bank's outsourcing and operational resilience guidance apply now.

How long should a pilot run?

Long enough to cover normal weeks and at least one busy period, such as a month-end. Several weeks is typical.

Key Takeaways

• Treat conversational AI as three decisions: the AI platform, the implementation partner and the network layer.
• Judge the platform on understanding, accuracy, disclosure and channels.
• Judge the partner on integration, handoff, regulatory evidence and ongoing support.
• Judge the network on availability, rerouting and whether AI runs where calls already travel.
• Map accountability across all three, and score a pilot on member outcomes before scaling.

Evaluating conversational AI for your credit union? Talk to DigitalWell's credit union team about a pilot scored on your own member calls.