Recently we talked about A/B testing and how it is used by every marketer to test certain campaigns and the response rate. Marketers will test out an A scenario vs a B scenario on consumers to measure the engagement prior to launching a full campaign that reaches their entire target market. It is an interesting and simple tool that allows marketers to see which wording, picture, theme, etc. is most liked by consumers between two options and using that information to target a specific market. However, this article in Wired called Your A/B Testing Isn’t Working Nearly As Well As You Think brings up valid flaws associated with the method. How do you know which variables are working and which aren’t? Every campaign encompasses numerous variables because in today’s digital world marketers aren’t solely focusing on one medium to communicate with consumers. A/B testing limits to just how to change one variable, but not necessarily how a campaign could be better without a variable all together.

This article suggests the concept of experimental design where a company launches a campaign that includes all the different possible variables such as promotion, email, format, layout, messages, etc. and then measured the response rates at the end and combined them with all possible combinations. They were able to identify where they were wasting time and money on variables that weren’t working and where they could focus for future campaigns.

I think this is a cool concept and as digital continues to grow and analytics tools continue to be created, experimental design may make A/B testing a thing of the past. Granted this process is still a little complicated, can be expensive, and can take time to implement, the results it provides are invaluable. Who knows where this will go next, someone just needs to create a platform that simplifies the process.