Hey Mikko, you are the data guy. Can you provide some insight for us?
This sounds simple, but what is insight? I had no idea, so I headed to my favourite resource on how to think better, the fs.blog.
What Do You Mean by Insight?
Everyone working with data or hiring someone to work on their data is looking for insights. Only few actually understand what they are looking for.
I’d be the first one to admit, that I did not have a good definition for insight. I was just happy that data was flowing smoothly from point A to point B, A usually being a data base of some it-system and B being a Power BI report. I’m not proud to say it, but I thought it was just the client’s responsibility to deal with insight.
It’s really important to understand the nature of insight so you know what you are actually looking for. Turns out that insight is pretty simple. But just like many things, simple does not mean easy.
Insight is something that changes how we think or act, and after, there’s no way to go back.
To rephrase, if you find insight, it will cause a change in you or in your organization. It’s true you can’t go back to a point in time where you did not have the knowledge, but it’s also true that you can always replace old insight with a new one.
Types of Insight
Insight is usually found in three different ways. Gary Klein has come up with a triple path model of insight.
- Contradiction
- Connection (Curiosity, Coincidences)
- Creative desperation
Contradiction means that you find an inconsistency and use it to rebuild the current perception. It makes you think “that can’t be right”. In his book Seeing What Others Don’t: The Remarkable Ways We Gain Insights Gary gives an example of contradiction using suspicious people that spotted the housing market bubble.
Connections means that you combine or put things together in a new way. An example would be that you are working on a problem in one domain and then hear a lecture about a completely different thing that then actually helps you solve your problem by connecting the dots. Curiosities and coincidences make you think that “what is going on here” and that leads to further investigation that can lead to insights.
Creative desperation means that when you are in a very bad situation you come up with a new strategy out of pure desperation. In Gary’s book one of the examples was how a person escaped from a forest fire in a mountain ridge by burning the ground in front of him so he could then take cover in the already burnt ground before the fire caught him (there was zero change to outrun the fire).
Designing the BI System
In the book Gary argues that the stronger design an IT system has, the weaker are the insights that come out of it. The main argument is that IT systems are designed with order and structure while insights are disorderly.
While I partially agree with him, that is not the full story. Probably you won’t gain the insight of how to escape forest fire by looking at BI reports, but you can still gain knowledge that changes behavior.
If we think about the data, then it’s true that BI systems and at least Power BI requires a fixed structure meaning a dimensional model that works best as a star schema. Even if the data model is fixed, we can still give the users flexibility with multiple dimensions, filters, visualizations, the option to change visualizations and allow the user to ask question about the data in natural language.
It’s also possible to enable access to the raw data for ad hoc reporting for a specific group of users. On an organizational level it’s best to have a single source of truth, but individuals in analytical roles should have access to the raw data to gain insights that can lead to more permanent BI solutions.
Only thing permanent is change and that holds true also for company goals. When the goals change, BI systems tracking the goal need to change as well.
Imaginary Real-Life Examples
Contradiction: a retail company assumed that a specific product category is popular, but looking at BI reports they realized that data contradicted their assumption. As a result, the company first decided to reevaluate their marketing and merchandising strategies for that product category, but ultimately their insight led to giving up that category.
Connection: an airline company noticed spikes every now and then in the customer complaints they received. By connecting this information with external weather data, the company was able to identify weather-related issues as a major cause of the delays. The company took some actions to mitigate the impact of the weather, but they also realized that they can’t control the weather. They reframed the problem and decided that the easiest way to reduce complaints was to inform the customers in advance about the possible delays.
Creative desperation: a manufacturing company was about to run out of money. They had high levels of inventory and low sales. People at the company were getting actious and the company went into full crisis mode. They quickly created a BI system to identify slow-moving products and offered aggressive discounts to clear the excess inventory. This not only improved their financial performance but also helped them reevaluate their inventory and product management strategy moving forward.
Are You Willing to Change?
Gary Clein has also made a superb point on insight and organizations.
“Organizations think they want insights and innovation and say that they want innovation, but in reality, they don’t, because insights force them to change”
Next time you ask me for insights, I will respond with a counter question.
Are you willing to make changes?
Conclusion
Insights play a crucial role in helping organizations make informed decisions and drive continuous improvement. By understanding the different types of insights and designing BI systems that foster exploration, collaboration, and innovation, businesses can unlock the full potential of their data.
Next time you seek insights, remember that they are more than just data points – they are catalysts for change, and it’s up to you to decide if you’re willing to embrace that change.