Business Experiments

This afternoon I’m returning to Columbia Business School to speak to the Economics of Organizational Strategy class about business analytics in the HR space. I’m excited to be back at my alma mater to talk about my work. For this class, the professor assigned The Discipline of Business Experimentation from the Harvard Business Review. From the summary:

The data you already have can’t tell you how customers will react to innovations. To discover if a truly novel concept will succeed, you must subject it to a rigorous experiment. In most companies, tests do not adhere to scientific and statistical principles. As a result, managers often end up interpreting statistical noise as causationβ€”and making bad decisions.

YES. All the buzz about big data and analytics fails to mention that if you are not testing correctly, your data may be driving you to make bad decisions! With businesses ramping up analytics capabilities in a big way, I think this happens much, much more than companies are willing to admit.

The example in the HBR article is about Cracker Barrel testing a switch from incandescent to LED lighting at its restaurants. At restaurants that installed the LED lights, traffic decreased. This would initially suggest that LEDs are bad for business. However, by digging deeper, executives realized that the LED lighting made the entrances look dimmer and customers turned away thinking the restaurant was closed. The LED lighting should have been brighter than the incandescent lighting, but individual store managers were going around the corporate lighting standards to install more incandescent lighting, thus making the store look brighter and more welcoming. Therefore, once these stores adhered to the new LED policy, they had fewer lights and were less luminous.

With all the attention around the tools and methods available with statistical analysis, I’m afraid this deeper digging may get short shrift.

It’s important that business professional not only dedicate more time to digging than analysis, but that we can speak the language of data and statistics. With so much more data available to businesses today, knowing how to use it for decision-making is a competitive advantage.

Advertisements