U of T researcher's website uses AI to help writers reach their audience

Many technology companies say they want to hire more women. But University of Toronto researcher Kawin Ethayarajh found the language and tone of most job postings in the male-dominated industry tend to be more appealing to men. 

“Even if the recruiter [or writer] didn’t realize it, they were implicitly writing things that would be more well-received by a male audience,” explains Ethayarajh, who is a master’s student in the U of T's department of computer science. 

His solution is Toasted, a website that uses artificial intelligence, or AI, to estimate how your writing will appeal to your audience and makes suggestions on how you can improve. It’s based on an algorithm that leverages a subfield of AI known as natural language processing – analyzing and retrieving information from language – to evaluate text for gender neutrality and audience. 

Ethayarajh says, on average, the distribution of female and male tones in language is equal, but the degree to which something is feminine or masculine depends on the context in which it appears – which is why Toasted picks out words used within a context. 

“For example, a word that is strongly male biased would be ‘squad’, because when used in an article or in copywriting, it tends to be surrounded by ‘these guys did this,’ or ‘the male sports squad did this,’” Ethayarajh says.  

“When [Toasted] deems a correlation of this sort to be strong enough, it highlights the term.”  

When Toasted is more confident in its predictions, it gives users alternatives for each highlighted word that would be more appealing to the audience they are trying to reach. Among the examples on Ethayarajh’s website is ad copy from Kylie Cosmetics by Kylie Jenner. The text is scored as “fairly effective” at speaking to a female audience – 57 per cent better than average.

Ethayarajh’s graduate studies research, under the supervision of Professor Graeme Hirst, includes understanding vector spaces and word embeddings, a technique used to help machines understand human language. He co-wrote a paper, “Towards Understanding Linear Word Analogies” with Hirst and Assistant Professor David Duvenaud that marks a major step towards better understanding these vector analogies. 

As for Toasted, users can also check how well their writing appeals to college students, retirees and others. A Google job ad for a software engineer, for example, is average for both men and women but speaks more clearly to a software engineering audience than it does to accountants. 

Ethayarajh says his goal is to add more audience groups. He would also like to define audiences by geography.

“Let’s say I’m selling a product that I think will do well in Southern California. Culturally, I want to use terminology and copywriting that will appeal to people there, more so than in other parts of the U.S.” 

So why did he name the website “Toasted?” 

“I took inspiration from a Lucky Strike ad campaign of the 1960s,” says Ethayarajh. “Toasted makes you think of a very warm, comfortable thing, so to me, toasted is the essence of what good copywriting can be.”