We Put Google’s New AI Writing Assistant to the Test

But its work began to look sloppy on more specific requests. Asked to write a memo on consumer preferences in Paraguay compared to Uruguay, the system incorrectly described Paraguay as less populous. It hallucinated, or made up, the meaning behind a song from a 1960s Hindi film being performed at my pre-wedding welcome event.

Most ironically, when prompted about the benefits of Duet AI, the system described Duet AI as a startup founded by two former Google employees to develop AI for the music industry with over $10 million in funding from investors such as Andreessen Horowitz and Y Combinator. It appears no such company exists. Google encourages users to report inaccuracies through a thumbs-down button below AI-generated responses.

Behr says Google screens topics, keywords, and other content cues to avoid responses that are offensive or unfairly affect people, especially based on their demographics or political or religious beliefs. She acknowledged that the system makes mistakes, but she said feedback from public testing is vital to counter the tendency of AI systems to reflect biases seen in their training data or pass off made-up information. “AI is going to be a forever project,” she says.

Still, Behr says early users, like employees at Instacart and Victoria’s Secret’s Adore Me underwear brand, have been positive about the technology. Instacart spokesperson Lauren Svensson says—in a manually written email—that the company is excited about testing Google’s AI features but not ready to share any insights.

My tests left me worrying that AI writing aids could extinguish originality, to the detriment of humans on the receiving end of AI-crafted text. I envision readers glazing over at stale emails and documents as they might if forced to read Google’s nearly 6,000-word privacy policy. It’s unclear how much individual personality Google’s tools can absorb and whether they will come to assist us or replace us.

Behr says that in Google’s internal testing, emails from colleagues have not become “vanilla” or “generic” so far. The tools have boosted human ingenuity and creativity, not suppressed them, she says. Behr too would love an AI model that imitates her style, but she says “those are the types of things that we’re still evaluating.”

Despite their disappointments and limitations, the Duet features in Docs and Gmail seem likely to lure back some users who began to rely on ChatGPT or rival AI writing software. Google is going further than most other options can match, and what we are seeing today is only a preview of what’s to come.

When—or if—Duet matures from promising drafter to unbiased and expert document finisher, usage of it will become unstoppable. Until then, when it comes to writing those heartfelt vows and speeches, that’s a blank screen left entirely to me.