Bernardo Lares & Igor Skokan - Min. Human Bias in Marketing Mix Models using Meta Open Source Robyn

Summary

Bernardo and Igor from Meta discuss the impact of privacy regulations on digital advertising and introduce Project Robin, an open-source tool designed to modernize Marketing Mix Modeling (MMM). They highlight its use of advanced techniques like multi-objective hyperparameter optimization and randomized control trials to minimize bias and improve marketing strategy. The tool aims to democratize MMM, making it more accessible and transparent for businesses to optimize their marketing investments.

Takeaways

• 😀 The speakers are Bernardo Lares and Igor Or, both from Meta, with Bernardo being a Marketing Science Partner and Igor a mountain climber and recent participant in a five-day run in the Sahara.
• 🌐 Bernardo has lived in three countries and is colorblind, which he humorously suggests might be a slight disadvantage in data science.
• 🔍 They work in digital advertising at Meta, focusing on the changes in the advertising ecosystem due to new regulations like GDPR and CCPA, which empower people to control their data.
• 📉 These regulations and browser changes have led to challenges in targeting, optimization, and measurement in digital advertising.
• 📈 They introduced econometrics and marketing mix modeling (MMM) as statistical tools to understand marketing's impact, which have become more prominent due to the limitations of traditional digital measurement techniques.
• 🛠️ Project Robin is an open-source initiative by Meta that aims to modernize MMM by reducing bias and leveraging machine learning to create a continuous, granular analysis tool.
• 📊 Project Robin uses advanced techniques like multi-objective hyperparameter optimization, regularization, and trend and seasonality decompositions to improve model accuracy and reduce human bias.
• 🔧 The project encourages community involvement, with a GitHub repository, a Facebook group for discussions, and a website with success stories and resources.
• 📈 Igor emphasized the importance of randomized control trials (RCTs) for providing causal information and selecting the best models, which is a core feature of Project Robin.
• 🌟 Companies like Resident and Wheely have successfully used Project Robin to quickly implement and optimize their marketing mix models, demonstrating the tool's value for both digital disruptors and established companies.

Q & A

What are the challenges faced by the advertising industry due to recent changes in regulations and browser policies?

The advertising industry is facing challenges due to new regulations like GDPR, CCPA, and LGPD, which empower individuals to control their data. Additionally, changes in browser policies, such as the deprecation of third-party cookies, have impacted traditional methods of measuring, targeting, and optimizing digital advertising campaigns.

What is the role of the Meta Marketing Science team in the context of these changes?

The Meta Marketing Science team works with advertisers and agencies to understand the effectiveness of campaign elements and devise strategies to increase the return on marketing investment. They focus on adapting to the changes in the advertising ecosystem by exploring new tools and techniques in response to the challenges posed by regulatory changes and browser policies.

What is Econometrics and how does it relate to Marketing Mix Modeling (MMM)?

Econometrics is a statistical tool that uses regression analysis to understand the impact of marketing and non-marketing activities over time. It is used in Marketing Mix Modeling (MMM) to build regression models that help understand the proportion of media driven by different channels and to analyze business performance.

How does the traditional MMM approach differ from the contemporary approach promoted by Meta?

Traditional MMM is manually built by experts, slow, heavily biased, and time-consuming, allowing for limited model iterations per year. Meta's contemporary approach, using Project Robin, enables more dynamic, frequent model updates, minimizes bias through advanced techniques, and incorporates experimental data for more accurate results.

What is Project Robin and how does it aim to improve MMM?

Project Robin is an open-source package developed by Meta to enable semi-automatic Marketing Mix Models. It aims to minimize bias and improve the MMM process by using advanced techniques like hyperparameter optimization, regularization, and experimental data to provide more accurate and actionable insights.

What are the four pillars of contemporary methods implemented in Project Robin?

The four pillars of contemporary methods in Project Robin are mitigating bias in model training, selection, and decision-making. This includes using multi-objective hyperparameter optimization, automated trend and seasonality decomposition, regularization to handle multicollinearity, and clustering to select the best models.

How does Project Robin utilize experimental data to enhance MMM?

Project Robin incorporates experimental data from randomized control trials (RCTs) to provide causal information and measure incrementality, moving beyond traditional MMM's reliance on statistical correlations. This helps in selecting models that more accurately reflect business performance.

What are the benefits of using Project Robin for advertisers?

Using Project Robin offers benefits such as faster implementation of MMM models, more dynamic and frequent model updates, reduced bias, and the ability to optimize budget allocation across media channels based on experimental data and advanced analytics.

What future developments are planned for Project Robin?

Future developments for Project Robin include a Python wrapper for easier use by Python users, nested modeling for advanced users, forecasting and prediction capabilities, and a user interface (UI) through a Shiny app for non-coders to interact with the tool.

How can interested parties get involved with Project Robin and contribute to its community?

Interested parties can join the Project Robin community through the Facebook group for discussions, contribute to the GitHub repository where the code lives, and visit the official website for more information and success cases. They can also provide feedback and feature requests to help drive the project forward.