Search has been the juggernaut behind the rapid growth in advertising spend over recent years. The sector, dominated by Google, continued to grow in 2020 while spending in traditional media declined – and last year UK search spend grew by a huge 39%, reaching £11.7bn, according to data from the Advertising Association/Warc.

Competing with social media 

Search has achieved this success by providing brands with a highly cost-effective, targeted way to reach consumers, especially those in-market. But to date, it probably hasn’t been the first channel you would associate with creativity or inspiration. In contrast, look at recent developments at the social platforms that are Google’s primary competition for ad spend, such as Instagram, TikTok, Snapchat and Pinterest. All of these have ramped up their offerings for e-commerce brands with increasingly sophisticated tools that combine the best features of online and offline shopping.

If there seems to be a gulf between what search and social can offer marketers, though, that won’t be the case for long. Last month, Google launched multisearch (currently available only in the US), a tool with the potential to make shopping on its search platform a more intuitive and responsive experience – which is great news for brands and consumers alike. 

Searching across formats 

Multisearch lets users search using both text and images at the same time. That’s a big development, since images and text are good at communicating different types of information. A picture might be worth a thousand words, but it isn’t very good at expressing specific criteria. 

Let’s say you’ve seen a dress you love the look of, but wish it had pockets – a common frustration with a lot of women’s fashion. Multisearch would allow you to search for similar looking products using a photo of that dress and the word “pockets”. If you’ve spotted a sofa that’s the right style for your living room but too big, you could quickly find similar ones that are under 1.5 metres in length. And if you’ve come across a beautiful holiday destination that’s outside your price range, multisearch could provide suggestions of other places with a similar atmosphere and cheaper accommodation available. 

In each case, multisearch offers users the sort of guidance a store assistant might provide, without the limitations of physical retail. It can provide users with much more specific and relevant results than text or image-only search, and these are key factors for improving ad conversions.

Multisearch makes use of multimodal AI, a burgeoning technology that is able to understand data of more than one type: not just text and images, but video and audio too. It comes alongside Google’s Multitask Unified Model (MUM), unveiled last year, which the tech giant sees as central to making the experience of using search more helpful and streamlined. 

MUM improves the ability of Google search to understand complex questions and provide results broken down into categories, bringing the experience closer to asking a human expert for advice. Suppose you wanted to start playing a new sport from scratch, for example. Before now, to find out everything you need to know to get started could be an overwhelming experience. With MUM, you could ask the question “how can I get started playing squash?”; Google would understand that you might need information on equipment, clubs and fitness preparation, and could highlight relevant information linked to each of these considerations.

The opportunities for e-commerce

The other key feature of MUM is that it can understand and translate information in 75 different languages. This offers obvious benefits for anyone planning a trip to another country in which they don’t speak the local language, but also reduces barriers for e-commerce brands aiming to connect with consumers in other parts of the world. Both aspects of MUM promise greater utility and better results to users, again potentially boosting the effectiveness of ad campaigns.

Like all developments in digital media, multimodal AI presents challenges as well as opportunities for marketers. The businesses best placed to take advantage of it will be those with a strong understanding of their competitive landscape, the USPs of their own products, and how they fit into the lives of consumers.