The FINANCIAL — In recent years there has been an explosion in new digital survey tools. These tools promise organizational leaders an ability to gain internal feedback from employees in a more frequent and targeted way than the traditional paper “annual survey.” Leaders can now take an agile and responsive approach to dealing with issues — one that simply wasn’t possible a decade ago.
At Gallup, we love using pulse survey tools. (Naturally, we like the tools within the Gallup Access platform.) However, the spread of these tools means there are a lot of amateur survey writers out there writing — let’s be frank — bad surveys. It’s not their fault. They are doing their best on limited budgets and with limited knowledge.
But that also means many business leaders are making decisions based on inaccurate, biased and confusing results — while mistakenly thinking these decisions are data-driven! For this reason, a bad survey is often worse than no survey at all.
Here are some common ways people (and even some professional survey services) get pulse surveys wrong:
You write questions that confirm your own biases.
“We all know we have biases — but being aware of that doesn’t stop them from affecting our decisions. As a general rule, it is a bad idea for someone to write survey questions if they have a personal interest in a particular outcome. This can make them unconsciously write or arrange questions in an order that results in the answer they want. Additionally, if respondents feel they are being asked leading questions, they may try to manipulate the survey — further skewing the results.”
You ask questions that are confusing or irrelevant.
Even with the best intentions, survey writers can write confusing questions. A question may use terminology that respondents don’t understand (or have different interpretations of). A question may ask multiple things at once, making it hard to decide how to answer. All these factors can impact the accuracy of your results.
You ask too many questions.
If you haven’t taken a survey recently, you may have forgotten how much time it takes to complete one. Adding a few more questions to a survey can decrease the quality and number of responses. Survey questions can be unintentionally redundant, leading to frustration for both survey takers and survey interpreters.
You survey too often.
More than a third of employees (36%) report receiving surveys regularly, three or more times per year. The rise of pulse surveys has made it incredibly easy to send one out to employees. Although the research is mixed on when survey fatigue occurs, it’s safe to say that survey fatigue does happen — and survey administrators should be aware of this. One problem with too many surveys is that people feel like they don’t matter if they don’t see any action taken to address survey results.
You create benchmarks based on your feelings.
What counts as success? What counts as good enough? If your survey benchmarks are based on your own gut feelings, then your survey results may not be as accurate as you think. On a 5-point favorability scale, you may think 4 or “agree” is quite good. In reality, 4 may be average. We see a lot of companies combine response categories (i.e., what percentage of respondents “agree” or “strongly agree”) when reporting their results to make it look like more people are responding favorably to survey questions. If you want a truly exceptional (and therefore more meaningful) benchmark, pay attention to the 5s (“strongly agree”) alone.
Ideally, you want to be able to compare your results with industry or regional averages. For example, Gallup Access can compare your organization’s survey results with more than 200 industries, regions and functions.
You don’t connect survey data with performance outcomes.
You can learn a lot about your people through surveys. Nevertheless, as a leader, your responsibility is the survival, health and growth of your organization. When done right, surveys can help you figure out where to deploy resources most efficiently — and which initiatives are likely to move the needle on what matters most. It’s not just about “making employees happy.” Surveys should be validated against performance data to ensure you are focusing on issues proven to be related to business outcomes.
You assume correlation is causation.
Statistics is a complex science. When two results seem to trend in the same direction, they may be causally linked, caused by something related to both, mutually interdependent, or simply a coincidence. For example, when both survey results and business results trend together, without more rigorous research there’s no way to know which caused the other. Including a survey methodologist as part of your data strategy is key to getting the most accurate insights.
You don’t provide survey results back to the people who took the survey.
Only about half (55%) of employees who strongly agree that they expected to receive results from workplace surveys strongly agree they received results. Survey administrators should provide survey results back to respondents whenever possible. This shows that you take their input seriously, and that the survey didn’t go into the trash. It can also be a helpful level set. Instead of being disgruntled that their opinions were not heard or acted on, employees can see that perhaps their opinion was a minority view. Transparency builds trust.
You don’t take action on the survey results.
This is crucial: Gallup’s research has found that when organizations send out an internal survey and take no action in response, they have lower levels of engagement afterward. In other words, doing a survey with no follow-up is worse than doing nothing at all. And yet only 8% of employees strongly agree that their employer takes action on survey results. We recommend our clients develop action teams before issuing a survey so they are committed to action from the get-go.
You don’t use survey results as a conversation starter.
Don’t replace dialogue with data. Survey results should encourage productive and focused discussions, rather than shut them down. These discussions can generate new questions, introduce hard questions and (most importantly) hone in on the right questions in the decision-making process. Additionally, they should be used by managers to start fresh conversations with employees that focus on future growth.
Psst! There’s a Better Way
Survey design is a careful science that should be executed by professionals. A scientifically rigorous third-party survey (hey, like the ones you find in Gallup Access) provides a host of benefits:
employee confidence that their responses are anonymous
questions that are clear, unbiased and benchmarked against industry leaders
a survey built using scientific best practices
the ability to connect survey data with specific performance outcomes
freedom for leaders to focus on taking action instead of administration
In other words, the goal is not just a tool that works but a process you can trust.
When making the really big decisions for organizations (things like vision, values, culture, training, budgets and benefits), leaders need to feel confident in the objectivity and reliability of their results — because the right action begins with the right survey.
Discussion about this post