I asked a few questions of Owned Media Executive Josh Carty, to find out more.

(N.B. If you’re interested in marketing applications of AI, Econsultancy’s Supercharged conference takes place in London on May 1, 2018 and is chocked full of case studies and advice on how to build out your data science capability. Speakers come from Ikea, Danske Bank, Just Eat, Age UK, RBS and more)

Econsultancy: Josh, could you give us an overview of how you use machine learning for keyword categorisation and SEO at iProspect?

Josh Carty: We work with one of the UK’s largest online retailers on their organic search and performance content strategy. We’re interested in understanding how their customers search for their products online and how we can use these insights to inform their SEO and performance content activity.

Given the size of our client’s product range (spanning hundreds of thousands of items) and the variety of ways in which customers can search for them, there were millions of relevant search terms for us to consider. To transform this data into meaningful strategic insights, our first objective was to classify keywords into categories. It would be impossible to classify millions of keywords into a complex product hierarchy manually, so we leveraged natural language processing and machine learning to complete the task.

Using a training set of 15,000 manually-labelled keywords, we developed a classifier that categorises keywords with a high level of accuracy. This performance was achieved in part through natural language pre-processing. Search queries are stemmed to their root, common words are discarded, and sequences of words and word pairs are used to discriminate keywords between categories. To ensure our insights best align with our client’s internal operations, we classify keywords into their internal product hierarchy. This is composed of three top-level categories, 11 subcategories and 52 individual product classes.

With a way to classify keywords at scale, we have been able to provide unprecedented insights into our client’s organic online visibility. For example, by combining data on search demand around keywords with click-through rate and search position data, we can model what proportion of traffic our client is capturing by product category.

In modelling the relationship between organic position and click-through rate, it’s important to consider variation in customer behaviour. Online shoppers may, for example, be much more willing to look lower down the search results when looking for a new laptop than a new pair of shoes. Using our classifier with a category-level regression analysis of position on click-through rate, we created individual click-through rate models for each product class.

Combining these two powerful machine learning techniques, we offer an unprecedented picture of our client’s opportunities in organic search. It has been the backbone of our organic and content strategy this year, helping us identify content opportunities and prioritise SEO recommendations.

E: We’re seeing supervised learning used for various functions in ecommerce – what do you think are the real success stories so far?

JC: Machine learning offers enormous opportunities in ecommerce, from product classification to customer segmentation to product recommendation. Given the variety of tasks machine learning seeks to solve, it can be useful to distinguish between types of machine learning algorithm.

One such distinction is between supervised and unsupervised machine learning methods. In the supervised setting, an algorithm is provided with examples of the desired output, such as the categorised keywords provided as training data in our classification project. In the unsupervised setting, the algorithm draws inferences without such examples and are typically involved in clustering and segmentation tasks.

While both methods have great application in ecommerce, the wealth of labelled data, made available by retailers’ analytics platforms, has seen an abundance of supervised learning applications. One particular success is in the development of recommender systems. These systems, familiar to users of Amazon and other online retailers, offer product recommendations based on the purchase histories of customers. In a supervised setting, these may be generated from users’ past views, ratings and purchases.

E: What size of retailer should be looking at machine learning, and how hard is it to achieve?

JC: Online retailers are exposed to millions of customers, making it impossible for any team of analysts or marketers to act on them all effectively. Machine learning offers retailers the opportunity to scale both their insights and operations in an efficient way. Whether it’s clustering thousands of customers into segments, or making personalised product recommendations to its customers, even the smallest retailers can benefit from machine learning.

With a wealth of open-source projects and free documentation available, machine learning is becoming increasingly accessible to all industries. Technologies once only available to large companies, with specialist research teams, are now accessible to insights and analytics teams.

More on AI and ecommerce: