In the enterprise voice AI landscape, the gap between initial excitement and practical deployment reality has become increasingly apparent. We recently spoke with Kevin Wu, founder and CEO of Leaping AI, about his journey from management consulting at Boston Consulting Group to building complex voice AI workflows for enterprise customers. Kevin's unique perspective- having advised on GenAI strategy before diving into hands-on implementation- offers valuable insights into where enterprise voice AI stands today and what it takes to succeed at scale.

Leaping AI makes self-improving voice AI agents for businesses to automate repetitive phone calls at scale.

The Pre-LLM to Post-LLM Transition

Kevin's entry into voice AI came at a pivotal moment in 2023, when he was part of BCG's GenAI task force analyzing applications across industries. "What I realized is that GenAI has a big potential to disrupt the call center software market and just customer service in general," Kevin explains. This strategic view led him to leave consulting and dive directly into building voice AI solutions.

The transformation Kevin witnessed represents what he calls "a very clear cut" transition from the pre-LLM to post-LLM world. Previously, voice bots could only handle predefined intents and specific phrases. "You have to say specific things. Otherwise, it doesn't work," Kevin notes about legacy systems. The LLM revolution changed this fundamentally, enabling natural conversations and flexible handling of diverse use cases.

However, this transition created challenges for existing players. "A lot of the pre-LLM players have lots of difficulty adopting to the post-LLM world," Kevin observes, "because they just have a certain predefined architecture of their solution." The shift required not just technical changes but a fundamental rethinking of customer expectations—moving from 100% reliability within narrow parameters to broader capabilities with acceptable error rates.

Moving Beyond the Hype Cycle

Kevin's perspective on the industry's evolution is particularly insightful given his consulting background. "Two years ago, there were many enthusiastic adoptions of GenAI. I feel at this moment, while there is still enthusiastic adoption, we have arrived to a more realistic picture of what GenAI can or cannot do."

This maturation has revealed several key challenges:

• Hallucination management: Ensuring accuracy in customer-facing interactions

• ROI demonstration: Proving concrete business value beyond the initial excitement

• Reliability at scale: Maintaining human-like conversation quality consistently

• Integration complexity: Working with existing business systems and processes

"Now we're at the more engineering stage," Kevin explains. "We're beyond the hype and it's about building solutions that actually bring value versus just the hype."

The Hybrid Architecture Approach

One of Leaping AI's key insights involves knowing when to use LLMs versus more deterministic approaches. Kevin advocates for hybrid architectures that leverage the strengths of both technologies.

"An LLM is very bad at doing math," Kevin notes as an example. "If you ask an LLM if 12 plus 6 is equal to 36 divided by 2, both should be 18, it might tell you that it's not the same." Leaping AI's solution involves delegating such operations to traditional computer science modules while using LLMs for natural language understanding and generation.

This hybrid approach extends to broader workflow design. Kevin explains: "We actually have a hybrid approach where we have LLMs, but also modules that do deterministic operations work together where the LLM is able to invoke or kind of tell those other modules, like do this, right, and then come back with the result."

The Economics of Enterprise Voice AI

Perhaps Kevin's most provocative insight concerns when voice AI makes economic sense. His analysis suggests that voice AI primarily works for larger operations: "VoiceAI only, especially if you automate customer support and don't generate additional revenue... I think VoiceAI primarily makes sense for larger call centers. So anywhere where you have 500 calls or more, 20 people or more in the call center."

The reasoning is straightforward economics. Implementation costs remain relatively fixed regardless of scale, while benefits scale with volume. "If you're a small call center, you probably, in absolute numbers, really have, cannot really, like, get rid of people in a way where the overall ROI makes sense," Kevin explains.

This doesn't mean smaller organizations can't benefit from voice AI—rather, they should focus on value-adds like 24/7 availability, improved response times, or freeing staff for higher-value work rather than direct cost reduction.

The 80/20 Rule in Voice AI Development

Kevin's experience reveals a counterintuitive aspect of voice AI development. Unlike traditional software where development typically consumes 80% of effort and testing 20%, voice AI reverses this ratio.

"I would say with any given VoiceAI project, it's probably the opposite, where it's fairly quick to set up the VoiceAI agent, especially now that we have prompts... but then to perfect it in iterative fashion, to kind of get from 80 to 100 percent is actually 80 percent of the work."

This insight has profound implications for project planning and resource allocation. Success requires sustained engagement and continuous optimization rather than a traditional "deploy and forget" approach.

The Critical Role of Voice AI Managers

To address the ongoing optimization challenge, Leaping AI has developed what Kevin calls "Voice AI managers"—specialized roles within customer organizations. "These are people, young, motivated, willing to work with AI that become deeply familiar with our system, with prompt engineering," Kevin explains.

These managers serve multiple functions:

• Project coordination: Managing the technical implementation across departments

• Quality assurance: Evaluating call performance and identifying improvements

• Prompt optimization: Continuously refining agent behavior based on real-world performance

• Cross-functional liaison: Bridging IT, customer service, and business stakeholders

Kevin emphasizes that successful voice AI deployment requires organizational commitment: "Any organization deciding to do voice AI needs to go all in... deploying a voice AI system is probably one of the more complicated technical things an organization can do because it involves several different stakeholders."

Complex Workflows as Competitive Differentiation

Leaping AI has positioned itself around complex, multi-step workflows rather than simple single-prompt interactions. Kevin describes their approach: "We are more passionate about repeatable use cases that span several steps and also that have some of read-write operation from some kind of database."

A prime example is their wine merchant customer's sales process, which Kevin describes as "a brutally complex use case across like 16 different conversation steps." The key steps include customer identification, recording consent, authentication, needs assessment, order taking, quantity specification, wine recommendations, shipping and billing address verification, voucher application, order verification, and final system integration.

This complexity requires sophisticated orchestration but delivers "consistently at 70 to 80% automation rate" because it remains repeatable despite its intricacy.

Scaling Voice AI Operations

The challenge of scaling voice AI operations across multiple enterprise customers has led Leaping AI to develop several strategies:

Customer Empowerment: "Give as much power to the customer as possible," Kevin advises, including system prompt access and configuration capabilities.

Graduated Deployment: Rather than attempting comprehensive automation immediately, Kevin recommends starting with specific, high-ROI use cases and expanding gradually. One retail customer began with a single use case that delivered "four times the ROI of what we promised them," which built confidence for additional deployments.

AI-Powered Evaluation: Leaping AI is developing AI systems to analyze call performance automatically, identifying what went well and what needs improvement without manual review of hundreds of calls.

Setting Realistic Expectations

Kevin's consulting background taught him the importance of managing expectations. Early in Leaping AI's development, he made the mistake of promising 80% automation rates across all use cases. "While we've hit 80% in some use cases, it has not been across the board," he admits.

Now, Leaping AI sets more conservative expectations: "We're telling them, hey, we are pretty confident we can automate 40 to 50% of... your calls. But even those that are not able to be automated, will still categorize the issue and we will put that into your Zendesk system."

This approach recognizes that even partial automation can deliver significant value when combined with improved triaging and case management.

The Future of Enterprise Voice AI

Looking ahead, Kevin identifies two key areas of excitement:

Model Improvements: "A lot of the problems we're seeing is like lack of instruction following, not able to do math," Kevin notes. Better foundational models would reduce the need for complex workarounds and hybrid architectures.

Market Consolidation: Kevin anticipates opportunities for voice AI companies to expand into adjacent areas, potentially offering comprehensive call center solutions rather than requiring integration with legacy systems.

The vision is ambitious: "If a customer comes in, they want like efficient call center operations. You just go to one vendor, which hopefully at some point is leaping AI instead of having, you know, another vendor that they have to integrate leaping AI into."

Key Takeaways for Enterprise Voice AI

Kevin's insights from Leaping AI's enterprise deployments offer several important lessons:

1. Scale Matters: Voice AI economics work best for larger operations (500+ calls, 20+ agents)

2. Hybrid Approaches Win: Combine LLMs with deterministic systems for reliability

3. Plan for 80% Post-Deployment Work: Optimization, not initial setup, consumes most resources

4. Organizational Commitment Required: Success demands cross-functional coordination and dedicated management

5. Start Specific, Then Expand: Begin with high-ROI use cases before attempting comprehensive automation

6. Realistic Expectations Enable Success: Under-promise and over-deliver to build customer confidence

7. Continuous Evaluation Essential: AI-powered analysis can scale quality management

As voice AI continues maturing from hype to practical deployment, Kevin's experience with Leaping AI demonstrates that success requires not just technical sophistication but also realistic planning, organizational commitment, and a deep understanding of enterprise customer needs. The companies that thrive will be those that can bridge the gap between AI capabilities and business realities, delivering tangible value while managing the inherent complexities of deploying conversational AI at scale.