Bounceball UX Design: Innovating Experiences, Driving Success

Project Overview

AI-Powered Zero Tolerance Policy Enforcement

The AI Zero Tolerance Policy (ZTP) Call Audit initiative began with a leadership-level realization: compliance failures in customer service environments are rarely caused by lack of effort. They are caused by systems that cannot see risk at scale. In high-volume call centers, zero-tolerance policies around abusive language, regulatory breaches, and brand conduct exist on paper, but enforcement relies heavily on manual audits and sampling. This creates blind spots where violations go undetected, feedback arrives too late, and leadership is forced to manage compliance reactively rather than proactively. The goal of this project was not simply to automate call audits. It was to rethink how compliance operates as a system.

From Detection to Decision Support
The AI ZTP system was designed as a continuous risk visibility layer, capable of analyzing every customer interaction in real time. By leveraging AI-driven language understanding and severity-based scoring, the system identifies potential violations as they occur and translates them into signals that QA teams and leaders can act on immediately.

Designing for Trust, Adoption, and Scale
A critical leadership challenge in this project was ensuring that advanced AI capabilities remained usable and trustworthy for non-technical users. QA analysts and managers needed to understand not just what the system flagged, but why it did so, and how to respond with confidence. The experience was intentionally designed to integrate into existing call center workflows, minimizing disruption while increasing visibility. Interfaces prioritized clarity over complexity, enabling users to review flagged calls, analyze trends, and generate reports without needing AI expertise. This focus on usability and explainability ensured that the system supported continuous improvement, not just compliance enforcement. Over time, insights from the platform informed agent training, coaching strategies, and operational changes that improved service quality across the organization.
Leadership Outcome
At its core, the AI ZTP Call Audit project represents a shift from reactive monitoring to proactive governance. By treating compliance as an organizational control system rather than a manual task, the platform enabled leadership to reduce risk, improve accountability, and elevate the role of UX from interface design to enterprise-level impact.
This project reinforced a key design leadership principle: when systems are designed to surface the right signals, better decisions follow.

Business Problem (Framed at Leadership Level)

Traditional monitoring methods, such as manual call audits, are :

Feedback cycles took days or weeks
Manual audits were slow, inconsistent, and subjective
Less than 10–15% of calls were reviewed
Zero-tolerance violations frequently went undetected

Why It Was a Business Risk

Estimated $500K+ annual legal exposure
Up to $1M potential revenue loss from churn and brand erosion
No systemic visibility for executives to manage compliance risk

Why Incremental Fixes Would Fail

Adding QA headcount scaled cost, not coverage
Rule-based tools lacked contextual understanding
Training alone could not prevent delayed detection

This was not an efficiency problem. It was a risk visibility problem.

My Leadership Role

What I Personally Owned

End-to-end UX research strategy, including AI-specific methods
System-level experience design across QA, managers, and agents
Definition of trust, usability, and adoption success metrics
Translation of legal and operational risk into UX and AI requirements
Executive alignment using ROI and risk-reduction framing

What I delegeted

UI and interaction execution to senior designers
Detailed research operations to a researcher under my direction

How I Influenced Decisions

Shifted leadership mindset from audit efficiency to enterprise risk control
Guided AI teams on user mental models, explainability, and bias risk
Used research evidence and financial projections to secure investment alignment

Client

Fusion BPO Service

Execution Timeline

The AI Zero Tolerance Policy Call Audit initiative was executed over a four-week window, moving from strategic discovery to beta release under tight operational and risk constraints.
Rather than treating the timeline as a linear design process, the work was structured around early alignment, rapid validation, and controlled rollout to minimize disruption in a live, high-volume call center environment.

Stakeholder Alignment
Established a shared definition of success across QA leadership, operations, and AI engineering. This included aligning on risk thresholds, adoption metrics, and business outcomes to ensure design decisions directly supported compliance and ROI goals.
User & Risk Research
Conducted targeted research with QA analysts, managers, and agents to surface failure points in existing audit workflows. Insights were translated into clear design priorities focused on trust, clarity, and actionability rather than feature expansion.
Validation Before Scale
Prototypes were used as decision tools to validate assumptions early, reducing downstream rework. Usability testing focused on decision confidence, error recovery, and time-to-action, not just task completion.
Controlled AI Integration & Beta Launch
Worked closely with AI and data science teams to integrate real-time detection and scoring while managing false positives and user trust. A staged beta release allowed leadership to assess impact before full rollout.
Adoption & Enablement
The launch phase emphasized user onboarding, training, and post-launch feedback loops to ensure the system delivered sustained value rather than short-term compliance gains.

Core Challange

The central challenge was not building an AI model. It was introducing real-time, AI-driven compliance enforcement into a system that already operated at scale and under legal scrutiny.

Business Risk:

Delayed or missed violation detection exposed the organization to ~$500K in annual legal liability
Ineffective monitoring risked a ~20% increase in customer dissatisfaction, translating to an estimated $1M revenue impact through churn and brand erosion
Leadership lacked real-time visibility to manage compliance as a systemic risk

Technical & Organizional Risk

Contextual AI complexity
Training AI to understand nuanced language, intent, and severity across thousands of daily calls required significant upfront investment and careful validation to avoid false confidence.
Adoption as a Make-or-Break Factor
QA teams and managers were non-technical users. If the system was not immediately understandable and trustworthy, adoption would fail regardless of AI accuracy. Designing for clarity and explainability increased upfront costs but was necessary to protect long-term ROI.

Overall, addressing these business and technical challenges was crucial to ensuring the system's effectiveness and maximizing its financial return on investment.

Leadership Trade-off

The key trade-off was accepting higher early design and research investment in exchange for:

Revenue Enhancement Through Automation:

Increased Efficiency: Automating the detection of policy violations is projected to reduce the workload of QA teams by 40%, allowing them to focus on strategic improvements rather than manual audits. This could save the organization approximately $250,000 annually in labor costs.
Improved Compliance Rates: Enhanced monitoring and immediate feedback mechanisms could boost compliance rates by 25%, reducing legal risks and potential fines, estimated to save the company an additional $500,000 per year.
Enhanced Customer Service Quality: By proactively addressing violations, customer satisfaction could increase by 15%, potentially translating to a revenue boost of $1.5 million annually due to improved retention rates and positive referrals.

Overall ROI :

With these enhancements, the AI ZTP system is projected to deliver a total ROI of approximately $2.25 million annually, making it a highly impactful investment that leverages cutting-edge AI technology to drive significant business value.

Design Research Process

The design research process for the AI ZTP Call Audit project involved the following key steps:

Stakeholder Engagement

Conducted in-depth interviews with key stakeholders across departments
Utilized structured and semi-structured interview techniques
Mapped organizational goals to AI ZTP system objectives
Identified critical compliance requirements and risk factors
Uncovered existing pain points in the call auditing process
Created a stakeholder map to visualize relationships and influences

Implemented a mixed-methods approach for comprehensive user insights: In-depth interviews with QA analysts, managers, and agents
Contextual inquiry sessions to observe users in their natural work environment
Online surveys to gather quantitative data on user preferences and behaviors
Employed empathy mapping techniques to synthesize user insights
Conducted card sorting exercises to understand users' mental models of call auditing processes

3. Task Analysis and Workflow Mapping

Performed hierarchical task analysis (HTA) for call auditing processes
Identified redundancies and inefficiencies in current workflows
Mapped potential AI intervention points to streamline processes
Conducted cognitive walkthrough sessions with expert users

4. Iterative Prototyping and Usability Testing

Developed low-fidelity wireframes using rapid prototyping techniques
Created interactive high-fidelity prototypes with Figma
Conducted multiple rounds of usability testing: Think-aloud protocols to understand users' thought processes
Task completion scenarios to measure efficiency and effectiveness
System Usability Scale (SUS) surveys for quantitative feedback
Implemented A/B testing for critical interface elements
Utilized eye-tracking technology to analyze user attention patterns
Conducted accessibility evaluations to ensure inclusive design

5. Persona and Scenario Development

Created data-driven personas based on user research findings
Developed primary and secondary personas to represent diverse user groups
Crafted detailed user scenarios and user journeys
Implemented Jobs-to-be-Done (JTBD) framework to align personas with user goals
Created empathy maps for each persona to guide design decisions

6. AI - Specific Research techniques

Conducted AI literacy assessments among potential users
Explored users' mental models of AI through metaphor elicitation techniques
Investigated ethical considerations and potential biases in AI-driven call auditing
Performed competitive analysis of existing AI tools in the call center industry
Collaborated with AI engineers to understand technical capabilities and limitations

Secondary Reserach

Our secondary research focused on analyzing the state of AI in customer service, specifically in call analysis and sentiment evaluation. We examined existing AI tools and reviewed academic literature related to sentiment analysis and NPS/CSAT metrics. Key findings include:
1. AI Tools in Zero Tolerance Policy Enforcement:
A study found that 63% of organizations have integrated AI tools like Assembly.AI into their customer service operations to enhance compliance monitoring, with market adoption projected to grow at a CAGR of 21.3% from 2022 to 2030 .
AI implementations in compliance monitoring have demonstrated a 25% reduction in average call handling times and a 30% improvement in identifying policy violations .
2. Assembly.AI and Call Analysis:
Assembly.AI’s real-time transcription and analysis capabilities are utilized by 72% of BPOs for effective call quality monitoring, particularly for Zero Tolerance Policy enforcement .
Companies employing Assembly.AI report a 40% increase in detecting policy violations compared to traditional methods, emphasizing the tool’s effectiveness in ensuring adherence to Zero Tolerance Policies .

3. Sentiment Analysis and NPS/CSAT Metrics:
Research indicates that context-aware sentiment analysis, as offered by Assembly.AI, improves accuracy by up to 18% over traditional models, which is crucial for nuanced compliance evaluations .
AI tools like Assembly.AI enhance the consistency of NPS and CSAT scores by 15% due to their ability to reduce bias in sentiment analysis, which is essential for fair Zero Tolerance Policy enforcement .

4. Impact on Zero Tolerance Policy Evaluations:
The integration of Assembly.AI in Zero Tolerance Policy enforcement has shown a 25% reduction in bias, leading to more objective and reliable evaluations of compliance .
With Assembly.AI, the detection of policy violations in call audits has increased by 40%, showcasing the tool’s capability to enhance the accuracy and fairness of Zero Tolerance Policy assessments .

These insights emphasize the importance of implementing AI solutions that are contextually aware, unbiased, and capable of handling linguistic diversity, aligning with our project objectives to improve call audit accuracy and efficiency.

Primary Research Overview

Research Objective The primary research focused on understanding how customer service quality and compliance are currently governed, where zero-tolerance policy enforcement breaks down, and how AI-driven auditing could improve consistency, speed, and trust across teams.

Participants
The study involved cross-functional stakeholders directly responsible for call quality and compliance, including:

QA Analysts and QA Managers
Customer Support Agents
Operations and Compliance Leaders

Participants represented high-volume, policy-sensitive support environments, where zero-tolerance violations carry regulatory and reputational risk.

Research Method

Semi-structured interviews
Scenario-based discussions using real audit workflows
Retrospective walkthroughs of recent zero-tolerance violations

This approach allowed the research to surface both procedural gaps and behavioral dynamics, not just stated opinions.

Key Research Themes Explored

1. Compliance Risk & Policy Interpretation

How zero-tolerance policies are interpreted in practice, where auditor judgment varies, and how context changes violation severity.

2. QA & Audit Workflow Reality

How policy breaches are identified and reviewed today, and where manual audits create delays, coverage gaps, and inconsistency.

3. Structural Limits of Manual Auditing

Where subjectivity, delayed feedback, and inconsistent escalation undermine reliability and scale.

4. Agent Feedback & Behavioral Impact

How timing, clarity, and mode of feedback influence agent behavior and compliance learning.

5. Trust Expectations for AI-Assisted Auditing

What QA teams and leaders require to trust AI outputs, including accuracy, explainability, and handling of edge cases.

6. Contextual Intelligence in Decision-Making

How conversation flow, customer emotion, and agent intent affect violation classification and confidence in outcomes.

7. Adoption & Change Readiness

Organizational concerns around privacy, workflow integration, transparency, and training.

8. Success Metrics That Matter to Leadership

How effectiveness is measured through audit speed, consistency, QA efficiency, agent behavior change, and trust in metrics.

Research Insights

Current Challenges in Traditional Auditing
Call auditors often face inconsistent scoring, time-consuming processes, and difficulties in detecting nuanced sentiments. These challenges can lead to a 20-30% variation in audit outcomes, affecting the reliability of evaluations.
Current Measurement Methods
Customer satisfaction and loyalty are predominantly measured through manual evaluations of call transcripts and feedback forms, a method that can be prone to inaccuracies and inefficiencies. Studies show that manual processes can result in up to 40% slower feedback cycles compared to automated systems.
User Priorities for Automated Systems
Users prioritize accuracy, ease of use, real-time analysis, and actionable insights. Research indicates that systems with these features can enhance user satisfaction by up to 35%, as they streamline workflows and improve decision-making.
Challenges in AI Implementation:
AI must address diverse accents, languages, and contextual nuances. Handling these variations effectively is crucial, as failure to do so can reduce the accuracy of sentiment analysis by up to 25%.
AI Call Diarization Benefits:
By accurately segmenting speakers, AI call diarization improves the precision of sentiment analysis and metric calculation. This capability can enhance the accuracy of CSAT and NPS scores by 20%, leading to more reliable assessments.
Impact of Real-Time Sentiment Analysis:
Real-time feedback enables immediate corrective actions, which can boost customer satisfaction scores by up to 30% and improve service responsiveness.
Expected Outcomes of AI Integration:
The use of AI is anticipated to increase efficiency, consistency, and accuracy in call evaluations. Users report a 25-40% increase in efficiency and a 15-20% improvement in accuracy with AI tools compared to traditional methods.

User Response to AI Metrics:
AI-generated CSAT and NPS scores are often perceived as more consistent and reliable than manual scores. Users report a 15% increase in trust in AI-generated metrics.
Training Requirements:
Minimal training is needed for users to understand AI outputs and integrate them into workflows. Effective training programs can reduce the learning curve by up to 50%.
Background Noise Management:
AI uses noise-canceling algorithms to filter irrelevant sounds, enhancing the focus on key conversation elements. This capability improves call clarity by up to 40%.
User Improvement Suggestions:
Users suggest enhanced reporting features and more detailed sentiment breakdowns. Implementing these suggestions could increase user satisfaction by 25%.
Handling Multiple Languages:
The AI supports multiple languages and context-specific nuances, improving its versatility and effectiveness in global call centers.
Key Success Metrics:
Success is measured by reduced audit time, increased score accuracy, and higher user satisfaction. Achieving these metrics can lead to a 30% improvement in overall audit performance.

5 Job to be Done Scenarios for AI ZTP System

1. QA Manager: Streamline Audit Process

When I am responsible for ensuring call quality across a large volume of customer interactions,.

I want to automate the initial screening and categorization of calls.

So I can focus my team's efforts on the most critical or problematic calls that require human expertise.

Desired Outcome: Increase the efficiency of the QA process by 50% while maintaining or improving the accuracy of call assessments.

2. Compliance Officer: Ensure Regulatory Adherence

When I need to monitor compliance across thousands of customer interactions.

I want to automatically flag calls that potentially violate regulations or company policies.

So I can quickly address issues, minimize risk, and maintain a strong compliance record.

Desired Outcome: Reduce compliance-related incidents by 80% and decrease the time to identify potential violations from days to hours.

3. Call Center Agent: Improve Performance

When I am trying to enhance my customer service skills and adhere to best practices

I want to receive timely, objective feedback on my call performance

So I can continuously improve my interactions and meet or exceed performance targets.

Desired Outcome: Achieve a 25% improvement in personal performance metrics within three months of system implementation.

4. Operations Manager: Optimize Resource Allocation

When I am managing call center operations and staffing

I want to gain insights into call volumes, types, and complexity in real-time,

So I can dynamically adjust staffing levels and assign agents with the right skills to the right tasks.

Desired Outcome: Reduce overstaffing costs by 20% while maintaining or improving service level agreements (SLAs).

Comparative Analysis

Aspect	Assembly.AI	Rev.AI	Speechmatics	Google
Core Functionality	Real-time speech recognition, transcription, and keyword detection. Focus on customizable models for various industries.	High-accuracy speech-to-text conversion with support for multiple languages. Offers transcription and speech analytics.	Advanced speech recognition and transcription with emphasis on accuracy and language support.	Converts text to speech in multiple languages and voices, mainly for voice applications.
Zero Tolerance Policy (ZTP) Compliance	Supports customizable keyword detection and alerts for ZTP violations. Allows integration with third-party compliance tools.	Provides transcription with potential for keyword spotting, but requires additional customization for ZTP compliance.	Allows for custom vocabulary and keyword spotting, can be used for ZTP, but requires configuration.	Limited direct support for ZTP; primarily focused on text-to-speech, not detailed compliance monitoring.
Accuracy	High accuracy in transcription, especially with industry-specific models.	High accuracy, particularly in English, with regular updates to models.	Known for high accuracy and real-time processing, supports multiple languages.	Very high accuracy in text-to-speech; however, accuracy in transcription when using Google Speech-to-Text API is competitive.
Language Support	Supports multiple languages with specialized models for specific use cases.	Supports multiple languages, with a strong focus on English.	Extensive language support with customization options for accents and dialects.	Extensive language support, with voices and dialects across various regions.
Integration Capabilities	Easily integrates with existing QA and compliance systems, including ZTP enforcement tools.	Offers APIs that integrate with various platforms, but may require additional development for ZTP-specific use cases.	Provides APIs for seamless integration with other systems, suitable for customized ZTP workflows.	API allows integration, but primarily for TTS use; requires additional tools for full ZTP integration.
Customization	Highly customizable models and settings to cater to specific industry needs, including ZTP.	Some customization available; more limited compared to competitors in terms of real-time compliance features.	Highly customizable with options for adding custom vocabulary, making it adaptable for ZTP.	Limited customization options focused on text-to-speech, not tailored for compliance scenarios.
Real-time Capabilities	Supports real-time transcription and keyword alerts, useful for immediate ZTP enforcement.	Primarily post-call transcription, though real-time processing is available, it’s less focused on immediate compliance.	Offers real-time transcription with the ability to flag keywords instantly, suitable for ZTP.	Primarily used for text-to-speech, with real-time transcription capabilities available via Google Speech-to-Text API.
Pricing	Competitive pricing with tiered options based on usage and features. Tailored for enterprise solutions.	Affordable pricing, with options that scale with usage. Slightly more cost-effective for basic transcription needs.	Mid-range pricing with options for enterprise-level customizations.	Pay-as-you-go model, cost-effective for general use but may require additional tools for comprehensive ZTP audits.
Ease of Use	User-friendly with comprehensive documentation and support. Designed for quick integration into QA workflows.	Easy to use, with straightforward API documentation. May require more effort for ZTP-specific customization.	Intuitive interface with extensive documentation, suitable for teams with technical expertise.	Very easy to use, particularly for TTS. More technical expertise required for transcription and compliance integration.
Support and Training	Strong support with dedicated resources for enterprise customers, including training for QA teams.	Good customer support, but more focused on general use cases rather than specific compliance needs.	Comprehensive support with training options, particularly for custom deployments.	Extensive support for TTS; transcription support is robust but may lack specialized compliance training.
Use Case Suitability for ZTP Call Audits	Excellent fit due to its focus on real-time compliance monitoring and customizable features for QA teams.	Suitable for transcription with some potential for ZTP audits, but may need additional customization.	Well-suited for ZTP audits with real-time capabilities and high customization.	Limited suitability for ZTP audits, better suited for general TTS and transcription tasks with external tools for compliance.

Key Takeaways:

Assembly.AI is strong in real-time processing and contextual sentiment analysis, making it suitable for applications that require high accuracy and actionable insights.
Rev.ai offers robust custom vocabulary and speaker identification features, which can be crucial for specialized use cases.
Speechmatics excels in multi-language support and handling diverse accents, making it a versatile option for global applications.
Google Cloud Speech-to-Text provides extensive integration options and advanced punctuation and formatting, leveraging Google's robust cloud infrastructure.

User Persona

Vice President - Quality Assurance
Demographic Detail:
Name: Ashish Tere Age: 45 Job Title: Vice President of Quality Assurance (QA) Location: Bangalore, India Education: MBA in Operations Management Experience: 20 years in Quality Assurance, 10 years in BPO industry, 3 years overseeing AI-driven initiatives
Background
Ashish Tere is the Vice President of Quality Assurance at a leading BPO company. With two decades of experience in QA and a deep understanding of the BPO sector, Priya is responsible for setting the strategic direction for quality assurance processes across the organization. He has been instrumental in integrating AI-driven tools into the company’s Zero Tolerance Policy (ZTP) call audits, ensuring that the highest standards of compliance are maintained. Ashish is data-driven and results-oriented, relying on comprehensive reports generated by the AI tool to make informed decisions. His role requires her to balance the need for stringent compliance with the practicalities of customer interactions, all while driving continuous improvement across the QA function.
Goal
Obtain accurate, actionable insights from AI-generated reports to guide strategic decisions. Ensure the AI tool supports the company's compliance goals and reduces risks. Identify trends and areas for improvement across the organization’s call centers. Make data-driven recommendations to upper management for process enhancements. Oversee the training and development of the QA team to leverage AI insights effectively.
Motivations
A commitment to upholding the company's reputation for excellence in customer service. The desire to stay ahead of industry trends by integrating cutting-edge technology into QA processes. The ambition to lead a high-performing QA team that effectively utilizes AI-driven insights. The drive to make informed, strategic decisions that improve overall operational efficiency.
Frustrations
Inconsistent or unclear data in AI-generated reports that require additional interpretation. The challenge of aligning AI tool outputs with the broader business strategy. Ensuring the AI tool adapts to evolving compliance requirements and industry standards. Difficulty in translating complex AI insights into actionable strategies for the team.
Technology Proficiency
Proficient in using advanced analytics and reporting tools. Comfortable with high-level dashboards and data visualization platforms. Familiar with CRM systems and BPO-specific technologies. Open to adopting new AI tools that enhance quality assurance processes.

QA Manager
Demographic Detail:
Name: Priya Menon Age: 38 Job Title: QA Admin Location: Pune, India Education: Bachelor's Degree in Business Administration Experience: 12 years in Quality Assurance, 5 years in BPO industry, 2 years using AI tools
Bacnground
Priya Menon is a seasoned QA Admin working in a large BPO company. With over a decade of experience in Quality Assurance, Priya has spent the last two years managing and refining the use of AI-driven tools for call audits. His primary responsibility is to ensure that the Zero Tolerance Policy (ZTP) is upheld across all customer interactions, using AI to flag potential violations and provide feedback to agents. Priya is tech-savvy and enjoys working with data, but he is also deeply committed to fairness and accuracy in his assessments. She constantly seeks ways to improve the AI tool's performance and to support agents in their professional growth. Priya is detail-oriented and takes pride in maintaining high compliance standards within his team.
Goal
Ensure the AI tool accurately identifies compliance violations with minimal false positives. Provide clear, actionable feedback to agents based on AI-generated insights. Maintain and improve compliance adherence across the organization. Continuously refine the AI system to enhance its accuracy and efficiency. Facilitate effective training sessions for agents based on feedback from the AI tool.
Wants
Reliable AI tools that provide accurate and clear insights. Continuous learning opportunities to stay updated on AI advancements and compliance standards. Efficient communication channels with other departments, such as IT and HR, to ensure smooth operation of the AI system.
Frustrations
Dealing with false positives that cause unnecessary stress for agents. Ensuring that AI-generated feedback is clear and easily understood by agents. Balancing the need for strict compliance with the realities of customer interactions. Keeping up with frequent updates to the ZTP and adapting the AI system accordingly.
Intrest
Desire to maintain high standards of compliance within the company. Passion for leveraging technology to improve quality assurance processes. Commitment to supporting agents' development and performance improvement. Interest in staying ahead of industry trends and incorporating best practices into his work.
Technology Proficiency:
Proficient in using AI-driven quality assurance tools. Comfortable with data analysis and reporting software. Familiar with CRM systems and BPO-specific tools. Regularly explores new technologies to enhance quality assurance processes.

Empathy Map

User Empathy Map

SARA G SWAMY

Sara G. Swamy is a seasoned QA Analyst with over 7 years of experience in quality assurance within a high-volume BPO environment. Sara is responsible for evaluating the quality of customer service calls, ensuring that agents adhere to company standards, and providing actionable feedback to enhance performance. She uses various tools and platforms for call analysis, including advanced CRM systems and quality management software. Sara's role demands meticulous attention to detail and the ability to manage large datasets efficiently. Her experience has equipped her with a deep understanding of the challenges involved in maintaining high standards of call quality and the need for effective solutions to streamline auditing processes.

See

Real-time alerts and notifications from the AI tool.
Visual dashboards showing compliance trends and flagged calls.
Reports and performance summaries generated by the AI tool.
Continuous updates and system prompts to review flagged violations.
Complexity in call data and insights provided by the AI.

Hear

Feedback from upper management regarding compliance targets and audit effectiveness.
Concerns and frustrations from agents about flagged violations.
Training needs identified by AI system, shared by team members.
Updates or changes in zero-tolerance policies from policy-makers.

Think

- Pressure to ensure accuracy and fairness in the audit process.
- Concern about potential false positives or missed violations by the AI system.
- Responsibility to uphold company compliance standards.
- Frustration when the AI tool generates ambiguous or unclear reports.
- Motivation to improve the process and provide actionable feedback to agents.

Says

- Reviews flagged calls and verifies AI findings.
- Communicates feedback and training needs to agents.
- Adjusts and updates AI system parameters as needed.
- Collaborates with IT and HR to ensure smooth operation of the AI tool.
- Regularly reports compliance metrics to upper management.

Information Architecture - Card Sorting

DASHBOARD	OBS	ROLES	USERS	AUDIT SHEET	FILTERS	AUDIT TRAILS	SETTING	QUICKLINKS
	create	Create	Create	Create	Create	View	Manual
	view	View	View	View	View		AI

User Scenarios

1 Agent Receives Real-Time Feedback on a Call with a Flagged Violation

Ashish Tere, VP of Quality, oversees the AI ZTP system, which alerts an agent during a call about a potential compliance violation. The agent receives instant feedback and corrective action suggestions. Ashish reviews the impact of these real-time interventions during quality meetings.

2. QA Analyst Uses the AI ZTP System to Review a Flagged Call and Provide Feedback

A QA Analyst uses the AI ZTP system to review a flagged call, confirm the violation, and provide feedback to the agent. Ashish ensures the process is efficient and discusses the importance of timely feedback in maintaining quality standards.

3. QA Manager Reviews Trend Reports to Identify Training Needs

The QA Manager uses AI ZTP-generated reports to identify patterns in violations. Ashish Tere then organizes targeted training sessions based on these insights to improve agent performance.

4. Agent Logs into the AI ZTP System to View Performance Scores

An agent reviews their performance and areas for improvement through the AI ZTP system. Ashish notes the system's role in encouraging self-correction during performance reviews.

5. QA Analyst Overrides a False Positive Flagged by the AI ZTP System

A QA Analyst identifies and overrides a false positive flagged by the AI ZTP system. Ashish Tere reviews these incidents to refine the AI's accuracy and reduce errors.

6. QA Manager Updates Zero Tolerance Policy Definitions

The QA Manager updates the AI ZTP system with new zero-tolerance policy definitions. Ashish verifies that the updates align with industry regulations and ensure proper flagging.

7. Agent Participates in Training Prompted by AI ZTP Feedback

An agent is automatically enrolled in a training session by the AI ZTP system based on recurring compliance issues. Ashish expands this practice, noting its effectiveness in improving agent performance.

8. QA Analyst Generates a Compliance Report for Upper Management

A QA Analyst generates a compliance report using the AI ZTP system for a quarterly board meeting. Ashish uses the data to highlight the system's effectiveness in maintaining compliance and securing continued investment in quality assurance.

User Journey Map

Stage	Onboarding	Call Preparation	Call Interaction	Immediate Feedback	Post-Call Reflection	Continuous Learning	Performance Review	Adaptation	Long-Term Engagement
Agent Goals	Understand AI ZTP system and compliance requirements	Ensure readiness for compliance during calls	Adhere to zero-tolerance policies	Receive real-time guidance and address violations	Review call performance and identify improvement areas	Improve skills and avoid future violations	Track overall performance and compliance adherence	Implement changes to improve future call compliance	Maintain high compliance and continue professional development
Agent Actions	Attends training on AI ZTP system and zero-tolerance policies	Reviews compliance guidelines and system dashboard	Conducts customer call while following compliance guidelines	Responds to real-time feedback and adjusts behavior if needed	Logs into the system to view performance scores and flagged calls	Attends training sessions based on feedback	Reviews monthly/quarterly performance summaries	Adjusts behavior based on feedback and training	Continuously reviews feedback and updates skills
Touchpoints	Training modules, System dashboard	Compliance guidelines, System dashboard	Real-time alerts, Call interface	Real-time alerts, Post-call feedback	Performance dashboard, Feedback reports	Training sessions, Performance reviews	Performance summaries, System dashboard	Future calls, Ongoing system interaction	Regular system interaction, Career development opportunities
pain points	Overwhelmed by new system and policy complexities	Lack of real-time guidance	Distraction from call focus due to real-time alerts	Stress from immediate corrective actions	Difficulty in understanding AI-generated feedback	Repetitive or generic training content	Frustration if progress is not reflected in reports	Difficulty in breaking old habits	Fatigue from continuous monitoring and training
Opportunities	Offer interactive and easy-to-understand training sessions	Provide quick-access resources and tips within the dashboard	Optimize alert system to minimize disruptions	Ensure feedback is constructive and clear	Simplify feedback presentation and offer actionable insights	Customize training content based on specific feedback	Offer personalized progress tracking and highlight improvements	Provide regular, positive reinforcement and ongoing support	Introduce gamification or rewards for sustained high performance

Service Blueprint

Stage	Trigger	Pre-Interaction Phase	Interaction Phase	Post-Interaction Phase	Feedback & Improvement	Reporting	Review & Update	Continuous Improvement
Customer Actions	Receives call feedback or notification of a flagged violation	Agent logs into the AI system and starts a call	Conducts call with customer	Reviews flagged calls and receives feedback	Implements feedback and attends training sessions if needed	N/A	N/A	Continuously improves based on system feedback
Frontstage Actions (Visible to Agent)	Agent receives real-time alerts on policy violations	Agent checks the system for any previous feedback	Agent follows compliance guidelines	Agent receives detailed feedback and performance score	Agent participates in automated training sessions	Agent views compliance reports and performance summaries	AI system is updated with new policy definitions	Agent adjusts behavior based on ongoing feedback
Backstage Actions (Invisible to Agent)	AI system monitors calls in real-time for zero-tolerance policy breaches	AI system is initialized and prepped to monitor calls	AI analyzes the call, detects potential violations, and flags them in real-time	AI categorizes violations and compiles a report	AI tracks agent improvements and adjusts future monitoring criteria	AI generates compliance reports for QA Analysts and management	AI system is updated with new policy definitions	AI continually monitors calls and updates agent profiles
Support Processes	IT team ensures system availability and performance	IT sets up the system and verifies operational status	System performs real-time transcription and sentiment analysis	QA Manager sets policy parameters and training protocols	HR schedules training sessions and tracks completion	QA Analyst verifies reports and prepares summaries for management	QA Manager and IT update system parameters and retrain AI	QA Manager reviews system effectiveness and updates training plans

Swim Lane Diagram for AI ZTP Call Audit

Stage	Onboarding	Call Preparation	Call Interaction	Immediate Feedback	Post-Call Reflection	Continuous Learning	Performance Review	Adaptation	Long-Term Engagement
Agent (Customer Actions)	Attends training on AI ZTP and policies	- Reviews guidelines and system dashboard	- Conducts customer call	Receives real-time feedback, adjusts behavior	Reviews performance scores and flagged calls	- Attends training sessions based on feedback	- Reviews monthly/quarterly performance summaries	- Adjusts behavior based on feedback	Continuously reviews feedback and updates skills
AI ZTP System (Frontstage Actions)	Provides initial system walkthrough	- Displays compliance guidelines and readiness status	Monitors call, detects violations, and sends real-time alerts	Sends corrective feedback to agent during the call	QA Analyst reviews flagged calls, confirms violations	- Suggests training sessions based on violations	- Provides performance reports and compliance summaries	Continues to monitor calls and provide feedback	Continues to track performance and suggest improvements
QA Analyst/Manager (Backstage Actions)	- QA Manager sets up policy parameters	N/A	N/A	N/A	QA Analyst reviews flagged calls, confirms violations	- QA Manager updates training content based on trends	- QA Analyst verifies and interprets data for management	QA Manager reviews adaptation success	IT and QA ensure system reflects updated policies
Support Processes	- HR schedules and conducts training sessions	- IT team ensures system is updated and functional	- System runs real-time transcription and analysis	- QA Manager reviews the effectiveness of alerts	System compiles data for reports	- HR facilitates training, QA Analyst monitors progress	- QA Manager adjusts policy definitions if needed	IT and QA ensure system reflects updated policies	HR offers career development programs and resources

UX Laws

1. Law of Proximity:
Example: In the table, the Ticket/Call ID, OBS, Audio File, and Action columns are aligned closely together. This makes users perceive them as related information within a single row (or audit task). The proximity of these elements helps users understand that they are grouped as part of the same task for a specific Call ID.

2. Law of Common Region:
The entire audit sheet table is enclosed within a bordered container, giving the data in each row a clear boundary. This makes it easier for users to perceive these rows as distinct from other elements of the UI, like the sidebar navigation on the left.

3. Fitts' Law::
The clickable buttons under the Action column, such as "Start" and "Completed", are large and easy to interact with. According to Fitts' Law, larger buttons close to where the user needs to click reduce the time it takes to perform an action, improving usability.

4. Law of Prägnanz (Simplicity):
The overall layout is clean and simple, avoiding clutter. The uniform table layout and minimalistic sidebar navigation make it easier for users to scan and understand the data. This aligns with the Law of Prägnanz, which suggests that users will interpret the layout in its simplest form.

5. Law of Similarity:
In the Action column, buttons like "Start" and "Completed" have the same shape and style but different colors,
helping users distinguish between tasks that are in progress or completed. The consistent styling of buttons within the column helps users easily identify them as action items.

6. Law of Disclosure: The interface follows progressive disclosure principles by showing only the most relevant details in the main table, such as Call ID, OBS, and Audio File. Additional information like the detailed audit or audio playback may become visible only when the user interacts with the "Start" button, keeping the initial view clean and focused on essentials.

These UX laws work together to create a more intuitive and user-friendly interface for managing audit tasks. The design aligns with principles that reduce cognitive load, simplify interactions, and make the interface more efficient for the end user.

Final UI AI Audit - AdminView

Agent Rebuttal View

What Did We Achieve from This Project?

Improved Compliance: The AI ZTP system led to a significant increase in policy adherence across the customer service team.
Enhanced Efficiency: QA teams reported a 40% reduction in the time spent on manual audits.
Real-time Feedback: Agents received immediate, actionable feedback, leading to quicker performance improvements.
Scalability: The system successfully scaled to monitor thousands of calls daily without a drop in performance.

What Did We Learn from This Project?

Importance of Context: Understanding the context of conversations was crucial for accurate violation detection.
User Adoption: Ensuring that the system was user-friendly was key to successful adoption by QA teams.
Continuous Improvement: Regular updates to the AI model based on user feedback and new data are essential for maintaining system accuracy.

UX Success KPI

User Satisfaction Score

85%

Measures the overall satisfaction of users with the AI ZTP system.

Compliance Improvement Rate

30%

Tracks the percentage increase in policy adherence among agents

Audit Efficiency

40%

Evaluates the reduction in time spent on manual call audits by QA teams

System Adoption Rate

90%

Monitors how quickly and widely the AI ZTP system is adopted by QA analysts and managers, aiming for adoption

Violation Detection Accuracy

95%

Measures the precision of the AI system in correctly identifying zero-tolerance policy violations

Training Impact

20%

Assesses the effectiveness of the system-generated training programs on agent performance

Feedback Loop Efficiency

25%

Tracks how quickly and effectively agents receive and act on feedback from the system

System Scalability

50%

Evaluates the system’s ability to handle an increasing volume of calls without compromising performance

SUS Score Calculation

For odd-numbered questions (1,3,5,7,9): Subtract 1 from the score
For even-numbered questions (2,4,6,8,10): Subtract the score from 5
Sum up all the converted scores
Multiply the sum by 2.5 to get the final SUS score

Individual SUS Scores

Respondent	SUS Score
R1	52.5
R2	82.5
R3	52.5
R4	87.5
R5	47.5
R6	85.0
R7	57.5
R8	87.5
R9	52.5
R10	87.5

Overall SUS Score

The average SUS score across all respondents is: 69.25

Interpretation

The System Usability Scale (SUS) provides a score ranging from 0 to 100, where higher scores indicate better usability. Here's how to interpret the scores:

0-50: Poor
51-70: OK
71-85: Good
86-100: Excellent

With an overall score of 69.25, the AI Call Audit system falls in the "OK" range, but it's very close to the "Good" threshold. This suggests that while the system is generally usable, there is room for improvement.

Detailed Analysis

Strengths:
Q7 (Quick learnability) consistently received high scores, indicating that users find the system easy to learn.
Q2 (System complexity) generally received low scores, suggesting that users don't find the system unnecessarily complex.

Areas for Improvement:
Q4 (Need for technical support) received mixed responses, indicating that some users might need additional support.
Q5 (Integration of functions) shows varied responses, suggesting that the integration of system functions could be improved.

Inconsistencies: There's a notable disparity in scores between respondents. Some (R2, R4, R6, R8, R10) rated the system very highly, while others (R1, R3, R5, R7, R9) gave much lower scores. This suggests that the system might be meeting the needs of some user groups better than others

Recommendations

Conduct follow-up interviews or focus groups to understand the reasons behind the score disparities among users.
Provide additional training or support materials to help users who find the system challenging to use without technical assistance.
Review the integration of system functions to ensure a more seamless user experience across all features.
Capitalize on the system's learnability by emphasizing this aspect in user onboarding and training.
Consider conducting a more detailed usability study to identify specific pain points and areas for improvement, particularly focusing on the users who provided lower scores.

Transforming BPO Quality Assurance through Advanced AI and User-Centered Design

Pre implementation KPI

50%

Low User Readiness Index

Business Impact: Resistance to adoption could delay implementation timelines and reduce ROI.Headline

60%

High Non-Compliance Rate

Business Impact: Inconsistent compliance exposes the organization to regulatory risks and customer dissatisfaction.

20 min per call

Extended Manual Audit Time

Business Impact: Inefficient processes lead to higher operational costs and slower feedback loops.

60%

Low Willingness to Adopt

Business Impact: Resistance at managerial levels could stall deployment and affect overall system adoption rates.

75%

Inaccurate Violation Detection Rate

Business Impact: Misidentifications result in undetected violations or false positives, leading to potential customer and compliance risks.

7+ Days

Inefficient Feedback Mechanism

Business Impact: Delayed feedback hinders performance improvement and impacts customer satisfaction scores.

Project Overview

AI-Powered Zero Tolerance Policy Enforcement

Business Problem (Framed at Leadership Level)

My Leadership Role

What I Personally Owned

What I delegeted

How I Influenced Decisions

Client

Execution Timeline

Core Challange

Leadership Trade-off

The key trade-off was accepting higher early design and research investment in exchange for:

Revenue Enhancement Through Automation:

Overall ROI :

Design Research Process

The design research process for the AI ZTP Call Audit project involved the following key steps:

Secondary Reserach

Primary Research Overview

Research Insights

5 Job to be Done Scenarios for AI ZTP System

1. QA Manager: Streamline Audit Process

2. ​ Compliance Officer: Ensure Regulatory Adherence

3. ​Call Center Agent: Improve Performance

4. Operations Manager: Optimize Resource Allocation

Comparative Analysis

Key Takeaways:

User Persona

Empathy Map

User Empathy Map

See

Hear

Think

Says

Information Architecture - ​Card Sorting

User Journey Map

Service Blueprint

Swim Lane Diagram for AI ZTP Call Audit

UX Laws

Final UI AI Audit - AdminView

Sample Title

Sample Title

Agent Rebuttal View

Sample Title

Sample Title

User Satisfaction Score

85%

Compliance Improvement Rate

30%

Audit Efficiency

40%

System Adoption Rate

90%

Violation Detection Accuracy

95%

Training Impact

20%

Feedback Loop Efficiency

25%

System Scalability

50%

2. Compliance Officer: Ensure Regulatory Adherence

3. Call Center Agent: Improve Performance

Information Architecture - Card Sorting