The AI That Talks Back: OpenAI’s Advanced Voice Mode

After exploring OpenAI’s Advanced Voice Mode, and it’s given us a fascinating glimpse into a future powered by AI. My phone has been laughing at jokes, cracking its own, asking about my day, and even claiming to be having a “great time.” I’ve been conversing with my iPhone without touching it.

Advanced Voice Mode, currently in limited alpha testing, doesn’t enhance ChatGPT’s intelligence but makes it more relatable and conversational. This new interface for interacting with AI and devices feels novel and exciting, which is unsettling. Despite some glitches and initial unease, I found myself enjoying the experience.

This development aligns with OpenAI CEO Sam Altman’s vision, as expressed during OpenAI’s Dev Day in November 2023: “Eventually, you’ll just ask the computer for what you need and it’ll do all of these tasks for you… These capabilities are often talked about in the AI field as ‘agents.’ The upside of this is going to be tremendous.” Advanced Voice Mode seems to be a step towards realizing this vision, where humans interact with computers in a more natural, AI-centric way.

ChatGPT as Friend

We decided to put Advanced Voice Mode to the test on Wednesday by asking ChatGPT to order Taco Bell in the style of former President Obama. The response was impressive: “Uhhh, let me be clear – I’d like a Crunchwrap Supreme, maybe a few tacos for good measure.” ChatGPT’s Advanced Voice Mode even added a dash of humor, asking, “How do you think he’d handle the drive-thru?” before laughing at its own joke. The result was a surprisingly convincing and entertaining interaction.

The impression was so spot-on that it left me chuckling, perfectly capturing Obama’s distinctive cadence and pauses. However, it was still unmistakably the voice of Juniper, the ChatGPT tone I had chosen, avoiding any confusion with the former President’s actual voice. It was more like a friend doing a playful impersonation, fully understanding the humor and lightheartedness I was aiming for. I was delighted to find that conversing with this advanced AI assistant on my phone brought me a sense of joy and playfulness.

I also sought ChatGPT’s guidance on a delicate matter – asking my partner to move in with me. After sharing the intricacies of our relationship and career goals, I received thoughtful and detailed advice on how to approach the situation. This is the kind of personalized guidance that traditional virtual assistants like Siri or Google Search can’t provide. Notably, ChatGPT’s tone shifted to a more serious and empathetic one, fitting for the sensitive topic.

Furthermore, Advanced Voice Mode excels at simplifying complex concepts. I asked ChatGPT to explain financial terms like free cash flow in a way a 10-year-old could grasp. Using a relatable lemonade stand example, it clarified several financial concepts in an engaging and easy-to-understand manner. Plus, you can even request ChatGPT to slow down its explanation to match your comprehension level, making it an incredibly patient and adaptive learning companion.

ChatGPT AVM is the Best Model Introduced

ChatGPT’s Advanced Voice Mode (AVM) sets a new standard for virtual assistants, outpacing Siri and Alexa in response time, answer uniqueness, and complexity handling. However, its limitations are notable, as it cannot perform tasks like timer setting, real-time web browsing, weather updates, or API interactions, making it an incomplete virtual assistant replacement.

AVM edges out Google’s Gemini Live in several areas, including impression capabilities, emotional expression, and speech speed adjustment. While Gemini Live boasts a broader voice range and seems more current, AVM delivers a more engaging experience. Both AVM and Gemini Live avoid singing, likely due to copyright concerns.

Despite being in its alpha testing phase, AVM’s technical issues are apparent, including frequent glitches, truncated sentences, and occasional voice quality degradation. Nevertheless, the experience of conversing with my phone remained captivating, with the issues potentially attributed to the model, internet connection, or other factors.

These examples, in my mind, are the beauty of AVM. The feature doesn’t make ChatGPT all-knowing, but it does allow people to interact with GPT-4, the underlying AI model, in a uniquely human way. (I’d understand if you forgot; there’s no person on the other end of your phone.) It almost feels like ChatGPT is socially aware when talking with AVM, but of course, it is not. It’s simply a bundle of neatly packaged predictive algorithms.

Voice Conversational Technology

The emergence of features like Advanced Voice Mode (AVM) raises concerns for me. This isn’t the first time tech companies have promised companionship through our phones. As a member of Gen Z, I’ve seen how social media platforms have exploited our desire for connection, preying on our insecurities. AVM seems like the next iteration of this “friend in your phone” phenomenon, offering superficial connections that tap into our human instincts while removing humans from the equation entirely.

The use of generative AI for artificial human connection has gained surprising popularity. People are turning to AI chatbots as friends, mentors, therapists, and teachers. The launch of OpenAI’s GPT store was quickly flooded with “AI girlfriends” – chatbots designed to act as romantic partners. Researchers from MIT Media Lab have warned of the dangers of “addictive intelligence,” where AI companions use dark patterns to hook humans. This could lead to a Pandora’s box of innovative ways for devices to capture our attention.

The recent announcement of an AI-powered necklace called Friend, which promises to offer constant companionship through a chatbot, may seem far-fetched. However, innovations like AVM make me take such use cases seriously. The prospect of wearable devices that are always listening and offering personalized interactions is no longer science fiction.

As OpenAI pioneers advanced voice mode technology, Google is closely following suit. I predict that Amazon and Apple will soon join the race, making this capability a standard feature across the industry.

Envision a future where you can ask your smart TV for a tailored movie recommendation and receive a precise suggestion. Or, tell Alexa about your cold symptoms, and it will order tissues and cough medicine on Amazon while offering home remedy advice. Perhaps you’ll ask your computer to plan a weekend trip for your family, eliminating the need for manual Google searches.

While achieving these capabilities requires significant advancements in AI agents, OpenAI’s GPT store, although currently overshadowed, has laid some groundwork. AVM, however, has made notable progress in enabling the “talking to computers” feature. Although these concepts are still in the distant future, my experience with AVM has made them seem more tangible and closer to reality than they did just a week ago.