I’m an analyst in the world of employee listening. My mission is to assess an organization’s culture and its attributes based on employee feedback. I diagnose problems and offer strategies for paths forward.
We work in partnership with company stakeholders to create better workplaces. Along the way, we measure our progress with surveys that investigate perceptions of manager effectiveness; leadership vision; Diversity, Equity, Inclusion and Belonging, and more. We’re trying to identify an organization’s “true” state. We measure what matters, crunch the numbers, produce charts and graphs, and make comparisons. With every successive cultural measurement, we hope we‘ve improved.
We want higher scores. Always. And that’s our proof of progress.
The Problem: Your Scores Aren’t Changing Significantly
Variance. Without it, we can’t tell one thing from the other.
LeadershipIQ, a leadership training firm, conducted a study of 3,000 HR executives. Among those who indicated their organization surveys employees regularly, nearly half 47% reported that their survey scores haven’t changed significantly. Results like this make me ask, “What’s the point?”
What do we do when measurements show no change between Time 1 and Time 2? Do we conclude that in reality, nothing changed?
What about when a survey results in high scores across all items? Some might say that’s a good problem to have. But because they exhibit little variance, we have no indication of what to work on next. How do we make progress?
In both cases, perceptions appear to be positive and stable. But at the same time, negative behaviors like voluntary turnover persist. We must be missing something.
I'm forced to conclude that high, stable survey ratings may, at times, obscure the truth of an organization’s culture. What do we do when survey measures become detached from their ability to accurately measure employee engagement? We turn to someone smart for a possible answer.
Relatively Speaking
I’m not a physicist, but Albert Einstein was. He famously formulated the theory of relativity. Putting aside the nature of supermassive objects in space, relativity states that the laws of physics are the same everywhere. However, our ability to perceive them depends on when and where we are. Motion, or change, is relative to a frame of reference. If we’re sitting on a plane and turn the page of a book, the book does not appear to be moving at the speed of flight, even though it is.
This same phenomenon is at work in organizations. Every morning, people across an organization perceive its culture relative to yesterday. That is, the threshold for their positive perceptions gets set — and reset — every day. The bar keeps getting raised, and our ability to detect shifts at ever-higher altitudes appears to become increasingly lackluster.
Implications for Employee Listening
What has led to this insensitivity in our measurement of employee perceptions?
Scores versus ratings in survey results
The term “score” is an unfortunate misnomer. Engaging cultures isn’t a game in which we “put up points” or dunk on colleagues with lower scores. There are no winners and losers; we’re on the same team.
It might seem subtle. But aiming for “better scores” encourages behaviors among practitioners designed to help them win rather than succeed. To win, they will — in small ways — avoid anything that might lower their score.
Selective editing to avoid bad news
We encourage higher scores in several ways and not all of them are good. On one hand, we communicate, make changes, action-plan, and action-take. On the other, we seek to avoid unpleasant news.
One way we do this is to remove questions from surveys if we “feel” the result will be unfavorable. This is most common with respect to questions about compensation.
Consider this example. Let’s presume you’re paid fairly — according to geography, labor statistics, controlling for inflation, and everything else. And you feel good about it. You give “Compensation” a favorable rating on your Continuous Listening survey. Then you discover that your colleague who started a few months ago was hired at a level well beyond what you're paid after years of service. Read Loyalty Tax. Now, suddenly, you're not paid fairly. And it has nothing to do with money. Your paycheck hasn’t changed. You have. But nobody’s asking — so the score remains high.
Repeated use of the same measurement
Another way we encourage higher scores is by repeating items on successive surveys. This alone can keep high scores high but at the expense of valid measurement. Have you ever heard the phrase, “Teaching to the test?” Repeated use of the same items “trains” employees, perhaps in some small way, how to “correctly” answer them.
All of these approaches conspire to raise our survey’s ratings, but obscure our understanding of our employees’ needs, wants, and desires. When measures fail to reflect an organization’s reality, they’re no longer valid. Leaders rightly begin to question the efficacy and relevance of employee listening efforts. We know that employee listening is strongly associated with successful business outcomes, so how do we fix it?
Tips to Capture Accurate Employee Perceptions
Here’s some advice for your listening strategy to ensure you’re able to capture changes in employee perceptions as you continue to improve the employee experience. While by no means a complete list, some simple changes to survey design, survey administration, and results reporting can keep your listening strategy relevant.
Avoid overly precise results reporting
We often treat our survey results as though they emerged from controlled experiments using graduated cylinders. Reported results to the hundredth decimal place give us a sense of precision that isn’t there. We probably use decimals in order to help us believe we have more variance than we do. Whole numbers will do. It is easier for us to see when two measures are the same. Once we identify their similarity, it is easier to replace one or both with something more relevant.
Ask harder questions
Don’t be afraid to ask a question you need the answer to. If there is uncertainty about a subject, chances are the “favorable” ratings compiled from its answers will be few and far between. Don’t let fear of low scores prevent you from asking the right questions. Change your viewpoint on lower ratings. The truth is that you want to know what people need and want to help them get it. If they already had it, their ratings would be high.
Improve response rates
The best way to drive ongoing survey participation is to make it a part of the cultural fabric of your organization. When employees know that taking surveys is “what we do,” it becomes less of an added task and transforms it into an integral part of the employee experience. By having a regular cadence and clear communication from top leadership within the organization, employees are more likely to get involved and provide survey feedback. Additionally, by cascading and aligning communications before and during survey administration, companies can ensure there is a consistent story being shared across all levels of the organization.
Avoid blaming leaders for poor survey results
Managing employee perceptions is a leadership responsibility. But holding leaders to account for higher or lower scores fails to concede that factors beyond their control might influence ratings, and that relativity is at work within them. More importantly, it encourages chicanery at best and deception at worst.
The best way to get robust, trustworthy employee listening data is to measure what matters to people, ask good questions, and ask them often. Employee listening can and should lead to better understanding.
Seeing Beyond Your Survey Score
When it comes to understanding your engagement data, Perceptyx stands ready to help. Our consultants work with your team to ensure that you’re asking the right questions via the best listening methods tailored to the needs of your organization — pulses for quick feedback, recurring surveys for understanding changes over time, safety culture surveys to understand safety perceptions, and crowdsourcing to quickly co-create solutions — and then quickly convert these data-driven insights from our People Insights Platform into meaningful action planning.
To learn more about how Perceptyx can help your organization trailblaze its path toward listening success, schedule a meeting with a member of our team.