Scaling AI #1: Michael Tambe
Last week, we caught up with Michael Tambe, Head of Data Science of Amazon Advertising Field Sales, to discuss his views on some of the challenges and opportunities that data scientists face working with real world AI / ML solutions. Michael has been a data science leader for over 8 years, focused on building data science teams in go-to-market areas within sales, marketing, and pricing. Here’s the advice that Michael has for other business leaders looking to deploy AI solutions and the best practices on optimizing ML model performance.
Why do business leaders choose to implement AI solutions for their business?
Business leaders decide to implement AI solutions for their business in order to solve business problems. On the B2B sales side, there’s a set of business problems for sales in which business leaders are hoping to solve using AI. Which accounts do I go after? How do I segment them? How many accounts per AE? Which products do I sell to which person at what specific time? There’s a series of decision points that you would ideally be informed by data that, in theory, once you get enough of an understanding, AI can make those decisions better than a human can. On the B2B marketing side, it’s a bit different due to the business problems that you’re hoping to solve. How much money should I spend? What is the ROI of it? How do I optimize my spending? Again, it’s a series of decision questions where each of them can be informed by AI. Whether or not it is just simple metrics, exploratory data analysis, machine learning or full blown AI optimization, it all just depends on the business problem.
You mentioned that AI can make informed decisions more than humans can. Can you provide an example?
There are some select applications where AI can make better decisions than a human. Let’s take marketing for example. You want to run a marketing campaign on social media with a goal to optimize on conversions. Initially, you start with one creative to see what the process is like. Then, you have 2, 3, 4 creatives. After testing out these creatives, you start to tweak certain elements such as messaging, fonts, colors, or the graphics that’s within the ad. Eventually, you start to see the complexity increase. But, once you have over 70 different creatives, it is almost impossible to manage them all on your own. That’s where AI comes to play and can make those decisions for you to optimize on your marketing campaign.
What is the largest challenge that data scientists face and how can you overcome them when deploying new models within the business?
I’d say there are two primary challenges: identifying business problems for which there is organization support and knowing how to balance science and intuition.
To illustrate the first challenge, I’ll share the following. A recruiter contacts me about a “Head of Data Science” job about once a month and I often take the call because I like to see how the hiring manager (usually the VP of sales, marketing, or operations) is thinking about it. The job description I hear most often is, “We have a lot of data and we’ve only scratched the surface for how we can take advantage of it. We really need a leader to come in and tell us how to use it.” As a data scientist this might sound great. A blank canvas to define strategy and a senior leader in your corner. It’s not. It’s a trap. It implies the senior leader assumes most of their data is useful in its current form. Most likely they’re collecting the wrong data and/or not collecting accurate data, in which significant data cleaning will be required. This is a recipe for expectation mismatch. It also implies that the senior leader hasn’t thought much about the specific business problems to solve. This means that when you suggest a data science approach to a business problem, you might hit a wall of, “Oh...well, we actually already have a working process for that.” If I can have a specific conversation with a senior leader about specific business problems, their current approach, and where data science might help, then I know I’m working for the right person. If not, it’s an easy pass.
The second challenge is knowing how to balance science and intuition. When we, as Data Science leaders, find a business problem, we tend to apply the most scientifically sophisticated method that we can to the problem. Why use rules-based models when you can use machine learning? Why use machine learning when you can use deep learning? The right approach depends on where the model will be deployed. In the go-to-market space, our models often have an end user audience in the thousands, not millions. These are sales, marketing, and/or pricing professionals who have done their job for 10+ years. They probably don’t trust the data because data in sales and marketing is notoriously untrustworthy. So, if you take a highly scientific approach from day one, you risk taking data that your end users don’t trust, inserting it into algorithms they don’t understand, and using the data to tell them how to do their job. You’re just going to hit a brick wall.
Instead, the better approach is to start simple and take your end users along for the journey. Do this first by using simple and interpretable models like multi-variable linear regression or decision trees to prototype simple models. My goal is not to provide novel insight, but confirm patterns in the data they intuitively understand. For example, if I’m building a segmentation model, having the model show relatively obvious things like “the larger a company is the more they are likely to spend”, and “the more they engage with our marketing, the more likely they are to buy.” Once I show about 5 of these “obvious insights” from the data, I can then ask the question of how to balance these different factors: something humans aren’t great at. “So company X is a little smaller than company Y but engages with our marketing more. Which is a higher priority?” The stakeholder usually can’t answer that and it naturally sets the stage for Data Science. That’s a quick way to get a v1 model that provides the value and foundation to add more complexity to your models.
How has data science guided the business around pricing for go-to-market?
So, there’s a subset of people that are in charge of pricing who use a survey-type method and insist that it can never be done by data science. But in theory, what you're trying to do with pricing is predict the demand elasticity. There’s a lot of ways to do that through testing. You can also do that through selective discounting. You start with a list price and offer some people 10% off, others 20% off, and then 30% off. Then, you start to get a sense of how people respond to different price points. This isn’t perfect due to the psychological effect of telling the customer the price is $7 versus $10, but with 30% off, people react differently to that. You structure pricing into a data science problem. Then, you build confidence in the data to deploy a model. If you can, you would preferably want to test it but that is not always easy.
Finally, do you have any advice for other leaders looking to implement AI for their business?
Be very, very, very clear on the business problem you're trying to solve and be very, very, very clear on the way you're going to measure it. That's literally it.