Case study
Citibank · Senior product manager, conversational design · 2022–2024
Conversational Banking Assistant
Raised chatbot containment from 40% to 65% in six months across 5.4M yearly interactions, saving $3.6M in the first year.
The problem
When I joined, the chatbot resolved only about 40% of its 5.4 million yearly interactions. The other 60% went to live agents, phone calls, or branches. Requirements changed constantly, feature refinement stalled, and releases slipped.
The cause was process, not technology. The people who owned each area (payments, disputes, money movement) weren't involved. Legal and compliance reviews stalled because the product team couldn't answer specific questions, so requirements kept changing mid-development.
What I did
Found where the bot was failing
I started with the metrics: the highest-volume intents and how well the bot contained each one. Then I worked with the data science team to cluster key topics. That showed coverage, gaps, and leakage: topics the bot covered so poorly that customers ended up with a live agent.
Brought the area owners in from discovery
I reached out to domain leaders and internal contacts to find the owners of payments, disputes, and money movement. Each area brought in its lead product manager and development lead from the start. Together we mapped the experience and its pain points. Because the details were settled up front, legal and compliance approvals came the same day, by email instead of meetings.
Designed for what customers meant
This was before large language models, so every conversation ran on a decision tree. Does "dispute" mean "I want to dispute a charge" or "what's happening with my dispute"? We built clarifying menus that got customers to the right answer without overwhelming them, plus live data connections to show and manage payments and disputes.
Knew when to send customers elsewhere
Some tasks, such as wires, need too many disclosures for a small chat window. For those, the bot sent customers straight to the right screen in the mobile app instead of forcing the task into chat.
Protected the goal during a platform migration
Moving from on-premises servers to a private cloud took much of the team's development capacity. I set each sprint's planned features against the migration work and reprioritized by effort and impact, so both kept moving.
The outcome
Containment rose from 40% to 65% within six months of the new strategy, and operational savings reached $3.6M in the first year. Adding chat entry points to key digital flows grew users by 20%.
I also added thumbs-up/down feedback so we could track sentiment by topic in real time. I mentored two junior PMs: one went on to lead the AI/ML product team, and the other took over the chatbot after I left.
Fix the process before the product. Containment stalled because the area owners weren't involved, not because of the technology. Bringing their product and development leads in early ended the requirement churn.
Look for leakage, not just gaps. Topics the bot covered poorly mattered as much as topics it didn't cover. Clustering with data science is what found them.
Not every task belongs in chat. Sending wires and similar tasks straight to the right app screen helped customers more than forcing them into a small chat window.