What are the Design Secrets behind AI Voices?

It would be quite surprising to hear about anyone wanting to make AI voices sound like robots. The human voice is so deeply ingrained into our culture, in our speech patterns, that sounds unnatural to us would be unnerving.

Yet, with the proliferation of Artificial Intelligence technologies, voice assistants have been developed that are not exactly human, but also not exactly robots. Voice assistants like Alexa and Cortana, designed to make user interactions more seamless, have a wide range of nuances and levels of customization that make them more human than just artificially manufactured responses.

I first became aware of these voice assistants in 2010, when the team at Microsoft created their voice command system for Windows 8.

The product was developed by the people at Microsoft’s Speech and Hearing Research Group, and the design was based on existing research from the team, which has been developing artificial voices since the 1990s. Over the years, researchers have improved voice recognition technology and created models of speech that are sound and speech-generating with the capacity to generate almost any sound.

As technology has developed, researchers have identified additional needs and weaknesses. In 1995, for example, the Natural Language Processing Research Group at AT&T’s Bell Labs conducted a comparative study of telephone telephone calls with and without “robotic speech,” and concluded that the “goal of the industry should be to build a telephone system that is capable of expressing a natural conversation.” It is clear that no voice system is perfect, and that the best research has room for improvement. Yet, there are characteristics of the artificial voices of today that stand out:


In 2013, a research team from Cornell University and Microsoft Research created an artificial speech system that uses language to create comedic dialogue. The system could reproduce well-known lines from “Forrest Gump” and “The Social Network,” but it also produced comedic moments by incorporating not just the word “comedy,” but also humor in specific words and phrases.

AI Customer Service Call Centers

Compared to the conversation you might have with a customer service agent, a conversation between a customer and a company’s artificial intelligence (AI) agent might seem friendly, impersonal, and even cold. Like a customer service representative, an AI voice tends to operate behind a wall, relying on customer data to determine the customer’s response. (Once the data is obtained, it’s often stored remotely to the company’s server. The agents are then guided to go through the appropriate information on the phone.)


AI’s most exciting aspect in this conversation may be the relief it provides to customers. An artificial intelligence customer service agent will not waste time repeating questions and responses that have already been stored in a database. If AI has to remember everything the customer says, there’s nothing left for the customer to remember. That may be the reason companies rely on the call center for customer service.

It might be a little bit off-putting for customers to be met with a bot upon trying to voice their concerns or get a problem they have sorted out, but once they get used to the fact that this is the more efficient way, they’ll jump on board willingly.

Share on facebook
Share on twitter
Share on linkedin
Share on pinterest

Contact me

If you’re looking to commission a piece or would like to discuss anything you’ve read on my blog please get in touch. I’m also open to accepting guest posts so if you’re an artist or a writer and this interests you hit the button below.