If you are interested in learning more about organizational response specific to COVID-19, there are several articles on our blog with insights about helping employees navigate this unprecedented situation.
Employee surveys will typically be made up largely of quantitative items, meaning closed-ended items in which the points in the scale can be assigned a numeric value, allowing psychometricians and statisticians to analyze the employee survey data to find useful insights. These items are great for zeroing in on where employees may face barriers, but for richer employee engagement survey analysis, qualitative (open-ended) comment data explains why those barriers are a problem and fleshes out information about the trends revealed in the quantitative data.
When measuring and improving the employee experience, quantitative data will tell us WHAT employees are experiencing, but qualitative data will get us to WHY behind the experience and how to improve it. Qualitative data gives detail to insights, but there is one caveat—because of its nature, it is more difficult to accurately analyze. In this article, we’ll discuss three best practices for survey design incorporating qualitative items. We’ll also examine employee engagement survey analysis of qualitative data, and employee satisfaction survey analysis methods used for integrating qualitative and quantitative data to present an accurate picture of employee sentiment.
Qualitative data analysis is most effective in the context of continuous listening. Our free guide, Continuous Listening: Developing The Right Strategy For Your Organization, will help you develop an appropriate strategy to fit your needs.
Design Your Survey To Provide The Right Qualitative Data
Effectively analyzing qualitative data begins with good survey design. The focus should be on getting the information stakeholders need so they will have the necessary insight to make smart strategic decisions. These are the three best practices for incorporating open-ended questions to elicit qualitative data:
1. Understand from the outset what the organization wants to learn.
When designing a survey, think about the back-end analysis so you know how to structure and code items and are aware of the key themes in play. Having this focus is the difference between a thoughtful, targeted approach and a fishing expedition; it also ensures that the analysis of the data is more likely to be of value strategically.
Keep in mind that you typically don’t have to gather all the data at one time. Employee engagement survey statistical analysis of quantitative data will help define what to ask about in open-ended questions. Qualitative data analysis works best in the context of continuous listening, where there is a baseline measure to pinpoint areas where a closer look is warranted.
For example, an organization’s quantitative data shows problems related to resources; “I have the resources needed to do my job” consistently is rated unfavorably by employees. On the next survey, responses to an open-ended question, “What resources, if any, do you need to do your job that you do not have?” zeroes in on the specific problem related to resources. Qualitative data is immensely helpful in action planning because it pinpoints the precise problem.
2. Align employee survey qualitative analysis to the organization’s strategic objectives.
There can be a temptation, for executives and even CEOs, to want to read all employee comments. Not only is this not a good use of executives’ time; it also muddies the waters in regard to what is most important, since the data is unstructured and not connected to meaningful data that allows leadership to determine what and where improvements can be made.
Like all data, qualitative data is more meaningful when it is analyzed and sorted by themes, and characterized by how positive or negative it is and how it relates to the organization’s strategic objectives and specific groups of employees. A simple read-through of unorganized comments has no framework and makes it difficult to separate the signal from the noise. Aligning the analysis of qualitative data to the strategic goals of the organization helps keep the focus on what matters most.
3. Ensure you have the right number of qualitative survey items.
Be strategic about the number of open-ended items you include on the survey, and with what quantitative items they are associated. If you have too many, the survey will take too long to complete. Getting feedback is important, but it is better to develop a focused survey that takes 10-15 minutes than an unstructured exploratory survey that could take much longer. A better approach is to focus instead on including items that will provide deeper insight into responses on quantitative items. To limit the amount of time needed to complete the survey, include fewer open-ended questions or limit the number of characters allowed for the response. Remember: Your quantitative items will give you a signal on what is happening, and your qualitative items will provide a signal on why these things are happening.
Your quantitative data will provide you with an understanding of where employee pain points exist, and your qualitative data will help you answer why. (Tweet this!) We recommend that organizations choose a small number of focus areas, meaning the types of qualitative items you will need will be relatively small. In the first point above, we mentioned how you could use a qualitative item designed to understand the “why” behind employee pain related to resource enablement, but let’s think about this more generally.
What if we don’t know exactly what we want to ask about? In this instance, one successful strategy is to ask broad open-ended qualitative questions about the overall strengths and weaknesses of the employee experience such as “What do you like best about being an employee?” or “What improvements would make this a better place to work?” respectively. When paired with a well-designed set of quantitative items, we can then work with the data in the Perceptyx platform to focus in on employee groups where we found unfavorable feedback to specific aspects of the employee experience, and then isolate the open-ended data to search for themes get to the why behind the what. This will give you an initial idea behind the why, but we also recommend subsequent targeted feedback gathering to develop a stronger signal.
Following these three guidelines will ensure you get the most out of your qualitative survey data.
Employee Survey Analysis Of Qualitative Data
Not that long ago, analysis of open-ended comment data was a laborious process; someone had to read each comment and sort it according to theme and sentiment. These days, qualitative data can be quickly sorted electronically.
The Perceptyx platform incorporates robust syntax already built around common themes that we can apply to all open-ended comments. Sorting is done via a search for keywords or combinations of words, and script sorting classifies comments as positive or negative and assigns a theme to the comment, within the tool itself. This allows comments to be sorted very quickly and associated with quantitative data. Because electronic sorting eliminates human bias or judgement calls, the sorting is more accurate than if someone was reading and classifying every comment. This sorting methodology also provides an opportunity to see the degree to which employees are concerned about specific themes, and can be used to do a deeper reading if it fits into the research question you’re trying to answer.
Once all the qualitative data has been sorted and classified, it can be compared against other data, comparing for example, the responses of unengaged employees vs. those who are engaged, to see what is similar and what is different in the experiences of those groups and suggest interventions for improvement.
For example, if the employee survey analysis for the organization reveals low development opportunity scores, the quantitative data can be correlated to the comments, with favorable responses partitioned from unfavorable responses, and the comments for each compared. This will help answer the question of what is driving employee unhappiness on the development opportunity metric. The quantitative data may reveal the drivers of the discontent, but the qualitative data is more likely to provide an answer to why and what to do about it.
Action planning from qualitative data, tied to demographics, simply offers a clearer path to solving specific problems than quantitative data alone. If you don’t get all the answers you need on one survey, qualitative feedback gives you the opportunity to zero in for a more focused look on the next survey, with even more tightly targeted questions to elicit the specific information needed to address the problem. As part of an ongoing continuous listening program, the value of qualitative feedback is that it provides the spice or the cherry on top of the quantitative data you collect—and with careful question design, provides more definitive insight to inform strategic action.
See the way forward to higher engagement.
The Perceptyx platform gives you the flexibility to develop a listening strategy that fits the needs of your organization and identify the barriers blocking engagement. Combined with support from our analytics experts, our platform can help you keep your finger on the pulse of your people’s perceptions, so you can provide the support they need to become fully engaged. Get in touch to see how we can help your organization increase engagement—and profitability.