How Google Duplex Evolved to Also Call Businesses for You in Gemini
Google Duplex has evolved from a narrow, task-specific system to a unified architecture powered by Gemini. This article explores the transition from pre-LLM closed-domain models to modern foundation models, detailing how the technology now enables autonomous task completion in Tasks in Gemini App.
Tags
Quick summary
Google Duplex has evolved from a narrow, task-specific system to a unified architecture powered by Gemini. This article explores the transition from pre-LLM closed-domain models to modern foundation models, detailing how the technology now enables autonomous task completion in Tasks in Gemini App.
How Google Duplex Evolved to Also Call Businesses for You in Gemini
When Google introduced Duplex in 2018, it marked a pivotal moment in artificial intelligence by pioneering natural voice conversations between AI and local businesses over the phone. Nearly a decade later, this technology has transitioned from a specialized research project to a core component of Google's ecosystem, now integrated directly into Tasks in Gemini. This evolution represents a shift from closed-domain models to a unified architecture capable of handling autonomous task completion across a wide range of real-world scenarios.
From Closed-Domain Models to Unified Architecture
The journey of Google Duplex began before the invention of the transformer and modern language models. In its earliest iterations, the system was constrained to narrow domains. Engineers trained custom recurrent neural network models on anonymized, task-specific phone conversations. Without pre-trained world knowledge, these models relied on extensive training data for each specific task type, such as interpreting the rigid flow of restaurant bookings versus checking holiday hours. This approach ensured the system could navigate domain-specific logic but required building separate dialogue models for every new category.
As open-domain foundation models emerged, the architecture began to shift. In 2022, Duplex started employing LaMDA in live calls. These pre-trained models simplified comprehension and common-sense reasoning, allowing the system to handle unexpected conversational turns and tangents while remaining grounded in context. However, early generative models were less predictable and sometimes slower than current standards. To address this, Google built hybrid stacks that paired generative models for flexible phrasing with deterministic state tracking and strict policy guardrails.
The most significant leap occurred with the integration of Gemini. By shifting to prompting the system with free-text task descriptions, Google removed the constraint of fixed verticals entirely. This allowed models to define tasks simply via a prompt, enabling generalization to new task types and nuances—such as asking for window seating or extra space for a stroller—without building separate dialogue models from scratch. Today, the calling stack uses Gemini to unify multimodal understanding, planning, and dialogue generation, allowing the system to navigate multi-constraint requests and adapt when a receptionist suggests another slot.
Technical Hurdles in Live Telephony
Despite advances in foundation models, live telephony presents unique challenges that pre-training on standard web datasets alone cannot solve. Google Research highlights two primary hurdles that the current stack must overcome to function effectively.
First is pragmatic grounding for pre-call and conversational context. The web does not contain all the data needed to capture how calls unfold with busy merchants. Nearly a decade of experience with goal-directed calls helps the stack grasp pragmatic norms, starting even before the call begins. In practice, this means matching a free-text task with a business based on the historical context of past conversations. For example, the system can predict whether a task will work by knowing a specific vendor does not take reservations for large groups over the weekend. During the call, this context helps the model determine what makes sense to ask at a given moment, how to read polite indirect refusals, and how to keep calls brief and effective.
Second is the requirement for sub-second latency in noisy acoustic setups. Calls often run over compressed phone lines against loud backdrops, including clattering in kitchens, espresso machines, street noise, and side chatter. The stack is tuned for low-latency streaming inference so replies land at a natural cadence. Custom acoustic and conversation cadence models assist the core AI system with handling background noise and pauses, ensuring the agent stops talking when expected to avoid talking over the other party.
Designing for Both Ends of the Line
A critical principle shaping Duplex's design is that the system must work just as well for the business answering the phone as it does for the user. This philosophy has driven several key design choices that ensure respect for merchant autonomy and user experience.
Clear disclosure is mandatory. Every call opens by clearly disclosing that it is an AI agent—specifically, an automated system calling from Google. Businesses retain control; merchants can opt out of automated calls in their Google Business Profile settings or simply say "Please stop calling my business" during a call. Furthermore, Duplex requires no setup and zero adaptation. It works through the phone a business already answers, requiring no software installation, no booking system maintenance, and no staff training. Whoever answers can handle the call the way they would with any customer, even mid-rush with a line at the counter.
Additionally, the system aims to provide "one conversation, many insights." Details gathered on one call can answer many later questions, preventing businesses from being asked the same thing repeatedly. For instance, Google Maps can show details that Duplex gathers about businesses, such as happy hour specials, patio seating, pet-friendliness, parking tips, or weekly trivia nights.
Integration into Tasks in Gemini
The culmination of this decade-long architectural evolution is the launch of Duplex capabilities within Tasks in Gemini App. While the technology has worked behind the scenes across products like Google Search and Maps for years, its integration into a personal AI collaborator allows users to coordinate real-world chores directly.
Completing tasks like making dinner reservations or booking a hair appointment is time-consuming, and live phone conversations move fast, rarely following a script. By integrating Duplex into Tasks in Gemini, natural voice interaction becomes a seamlessly integrated modality that the agent can deploy whenever structured digital channels like online APIs or booking forms do not exist.
This integration leverages the newest Gemini 4 models, including Gemini 4 Hydrogen and Gemini 4 Neon. Through these capabilities, Tasks in Gemini can support more comprehensive workflows, such as analyzing utility bills, processing transactions securely via the Agent Payments Protocol, and coordinating with users over email and Rich Communication Services (RCS). This marks a significant step toward closing the loop on daily errands that require speaking directly with another person to coordinate details reliably.
Conclusion
The evolution of Google Duplex from a narrow, task-specific system to a unified stack built entirely with Gemini demonstrates the power of foundation models in solving complex, real-world problems. By addressing technical hurdles in live telephony and designing with both users and businesses in mind, Google has created a system that respects human interaction while automating tedious tasks. As these capabilities expand within Tasks in Gemini, they represent the next generation of AI assistance, bridging the gap between digital planning and physical execution.
Source: https://research.google/blog/how-google-duplex-evolved-to-also-call-businesses-for-you-in-gemini



