Making data-driven decisions should start and end with data.
That much should be obvious. What might not be as obvious is that there is still a place for more traditional resources like expertise and instinct, even when working with data. Accepting data at face value, without taking a critical view of its history, might mean overlooking errors or missing context. The result is that even if your decision was driven by data, it still might have been misled.
Data literacy means understanding all of the tools and techniques used in data analysis. It should be obvious that making a data-driven decision requires a high degree of data literacy. But just as, if not more important, is data fluency – understanding how to communicate truthfully and persuasively using data. Striving for data fluency is essential to informing our decision-making, and a critical part of data fluency is developing the instinct for interrogating data.
Let’s discuss the three essential questions to ask when interrogating your training data, or any data, to drive decision-making:
- What data do I need, and how can I get it?
- Is my data reliable?
- Do I have all of the context I need to interpret my data?
Meta-Analyzing Your Data
To interrogate your training data, you need to ask questions about the data itself. A data interrogation isn’t really about what the data is saying. It’s more about taking a good look at why it says that, and whether it can be trusted.
You need to know that you’re looking at the right data, that that data is reliable, and that you aren’t missing important context. Only after you’ve made sure of those factors, can you make a decision that’s truly data-driven. Those three factors are the biggest drivers of utilizing data as effectively as possible. Let’s take a look at those factors in detail.
What data do I need, and how can I get it?
At the start of the decision-making process, interrogate your decision itself. What kind of data would you need to assess the situation and make an informed choice? What is the scope and importance of the decision? A decision with bigger impacts may deserve more investment in finding obscure but useful data than a less-important decision.
Also consider how clear-cut the data you gather might be. Some decisions can be easily made based on a few clear data points. A single A/B test is probably sufficient to decide between two potential graphic designs, for example. But a more complex and open-ended decision, like what content to include in a new set of courses, will require far more data, and it’s unlikely that any one data point will be decisive in making the decision.
Finally, ask how you might go about accessing relevant data points. Some of the relevant data might not be easy to access, or it might be outside of the training team’s usual purview. Having a game-plan for how you will go about acquiring your data makes gathering it easier, and also makes the next question easier to answer.
Is my data reliable?
When you start accessing data, it’s important to interrogate it carefully to make sure it is reliable. A data-fluent professional will be able to look beyond what the raw data is saying, and assess whether it can be trusted.
Some data is just naturally unreliable. A free-response feedback survey provides interesting information, but it isn’t a rigorous way of collecting data and can’t be treated as concrete truth. That isn’t to say things like free-response feedback surveys aren’t an important part of assessing something like instructor performance. But it’s important to keep in mind that they are often highly biased or subjective.
Even data gathered through reliable methods can accumulate errors over time if it isn’t handled carefully. Something as simple as a typo during manual data-entry can cause mistakes. Incompatible software systems, all too common in L&D, can mangle your data when they try to interact. And of course, over time, all data sets become increasingly inaccurate compared to the present.
Making a good decision based on data requires taking the time to interrogate the reliability and quality of that data. Knowing how much you can trust your data is essential to knowing which factors you should weigh more heavily, and identifying where you need to improve or update your reporting.
Do I have all of the context needed to interpret my data?
Even relevant and reliable data might be missing crucial context that could cause it to be misleading when taken out of context. Training is a very complex function that engages with employees and processes from across the organization. Without full context, training data in isolation often fails to fully capture the impact of training operations.
Imagine trying to assess the effectiveness of vILT learning in 2021, using data from 2021 and 2020. Of course, a global pandemic forced an often-messy transition to vILT during that period. The data from these years would suggest that vILT is an extremely ineffective training delivery method, but only if taken out of context.
That context does not present itself in the raw data. It needs to be researched and investigated separately, alongside the data.
It’s common knowledge that essentially any workforce data from 2020 needs to be treated as an outlier. What other outliers are there in your data that aren’t so obvious? Only through a thorough interrogation of your data can you start to unlock that hidden context and assess what factors you may be missing.
The Importance of Robust Data Access
What’s the unifying principle behind these three data interrogation questions? What’s necessary to ensure that the data is relevant, reliable, and interpreted in-context?
Access.
In most training software available on the market, reporting is limited to pre-generated, non-customizable reports. They report on what the software developers believed would be important to you. That’s fine if your training team aligns perfectly with their assumptions, but what happens when you need to access data that isn’t on the report?
Short of digging through the raw inputs yourself, it’s often difficult or impossible to get commercial training software to output nonstandard data sets. And that’s just not going to cut it when you need to drive strategy with data.
What’s needed is more customizable access to training data. A comprehensive, meaningful reporting option that’s capable of providing you with the data that you need and putting you in the pilot’s seat for your own training data. That solution is Administrate.
Administrate’s no-code reporting engine provides the capability to customize your access to training data. By providing you with complete control over your own data, you can iterate through the interrogation process without worrying about whether your software can keep up with your needs. Reports don’t need to take days of work to assemble – with the analysis tools in Administrate’s reporting engine, complex custom training reports can be generated within minutes.
The ability to access the data you need is absolutely essential to your team’s success with data. To learn more about maximizing data access, and doing more to boost your team’s use of data, take a look at our Decision Economy Guide.
