Summary
Determining the extent of what you don't know improves the quality of your questions. Simply reviewing what you don't know about who, what, where, when, why, and how is a good start. Generating hypotheses as a brainstorm helps to formulate questions and analytics.
Knowing What You Don't Know
Try to understand the limits of your knowledge before you embark on data collection and analyses
When most people start to collect information, perhaps using surveys or interviews, they usually have a premise in mind that they want to confirm or deny. This is completely natural. Often we already have a good idea of what is going on. However, just because our assumption may be true, it doesn't mean that other issues aren't at play as well, issues that we haven't thought of or don't consider to be as important. As a result, the data we collect tends to be skewed or narrowly focused on our assumptions. This is why taking some time to consider the extent of your knowledge, and the possible extent of what you don't know, is important before you start on data collection. Here are two simple tools to help structure the process of thinking about the unknown.
The WWWWWH (What, who, where, when, why and how) table:
Problem: Urgent customer requests are not being addressed in a timely manner
What's known | What's not known | |
---|---|---|
What | Customers with are complaining about lack of responsiveness | Are the requests being lost, ignored, or forgotten? |
Who | Only customers with urgent requests | Is it the same customers who always have urgent requests? Are their non-urgent requests okay? Who is responsible for handling urgent requests? |
Where | At the corporate customer service center | Are requests given to sales reps okay? |
When | Only last two months | What's different from two months ago? New systems or procedures or staff? |
Why | The requests are not documented and not in system | Why aren't these requests documented and flagged? |
How | Urgent requests are usually handled manually, outside of the system and sent via email or by phone | What's the current procedure for handling urgent requests? How has it changed? |
Now when the customer service manager starts to grill the call center employees on what's going wrong, at least he/she will be armed with better questions, especially starting with "What changed two months ago?" The manager will also know to check with sales reps to see if they are having problems, as well.
Generating hypotheses - My best guess is....
My preferred method of developing good questions is to borrow from the scientific method and generate hypotheses for the causes of the problems. For those people who love brainstorming ideas or solutions, this is perfect. Gather a team of people to hypothesize why the problem is occurring and then design your data collection around proving and disproving these. A good set of hypotheses can help frame how much you don't really know about the problem.
Here's an example from a client whose new products were unsuccessful in the marketplace despite comprehensive marketing campaigns and well-executed introductions. Before we started any interviews, surveys or other analytics, the team brainstormed possible reasons why in a number of areas (poor quality, not meeting customer needs, price points not accurate, etc.)
Hypotheses for why new products are not meeting customer needs
- We don’t know our customers well
- We lack mechanisms and processes to communicate with customers
- There's no one responsible for global ownership of the customer relationship
- We have inaccurate forecasts of needs
- Head office incorrectly dictates products based on domestic customers
- Product introduction too slow to meet changing needs and competition
- We're not communicating with the right people in the customer organization
Going through these types of thought exercises helps you to ask better questions to find out what you don't know but need to know.