Search

Is your survey following all best practices?

Updated: Nov 28, 2019

Learn surveys with Meena Das - Part 1: Survey Design


P.S: We encourage our readers to go through the entire blog post, but should your reading time today be even limited than you expected, then please scroll down to the summary section at the bottom. Every minute you have for reading should empower your perspective for the minutes you are not reading.


Welcome to the first part of the mini-series Learning Surveys with Meena Das. This part is focused on the best practices of survey design.





Online surveys (or any survey for that matter) are one of those age-old topics that have immense amount of research available though books and internet. Have you ever thought – why is that the case? Or, why do we still have surveys in the world that are not following these best practices? Well, surveys are one of the oldest methods of research. Because the way they are set up to the way they are analyzed does not have any hard and fast rule in the back-end (something that mathematics formulas will never allow), every survey turns out to be different and most often not following the best practices. So, the question is, what could this blog post tell you that you may not know already? This mini-series is focused on how you can connect the objective of your survey with the analysis you produce out of that survey. To make that connection relevant, to make your survey successful, to ensure your survey is set-up to give you truly the quality of insights you are expecting out of it, it is really important to start with the basics – design principles of a survey based on caution and best practices in mind.


1. Understand the ‘why’ behind your survey

Your survey should have a purpose, a “why” that will motivate the design. The clearer you are in your objectives behind the survey, the better would be the questions you ask in the survey. Without a clear objective, surveys tend to deviate in any direction, and you risk losing the short attention of your survey respondents.


2. Ensure the length of your survey is not more than 5-7 mins

Once you have established the objective behind your survey, ensure that the questions you ask do not go into a rabbit hole. Your aim with your survey should be making the most efficient use of the little time your survey respondents are willing to give you. Ideally, 5-7 mins or 15-20 questions should be your goal.


3. Keep the wording of your questions simple

The wording of your questions directly impacts the quality of data you receive. Your wording should not be what is simple to understand for you but for your audience. Look for words that is common for your audience. This reduces the time at your respondents’ end to understand the question or imagine the scary scenario, when they do not understand but really mark an answer choice/leave blank just to complete your survey.


4. Avoid asking double-barreled questions

Double-barreled questions are when you ask for feedback on two separate things within a single question. Example, “How would you rate the quality of our customer support and the knowledge they have shared in solving your problem?”. This classic example of asking double-barreled questions can confuse your respondents when they have differing opinion about “overall quality” of your customer support vs “the knowledge” your customer showed. This confusion can trip the responses you receive, thus once again, putting in danger the quality of the data you collect.


5. Avoid leading and biased questions

Your choice of words (as described in point #3) is just as critical as how you phrase your questions. Any question that already reflects some positive or negative thought can be injecting bias in the minds of your respondents. For example, “Mango ice-cream is the most preferred dessert in Canada. How would you rate the quality of mango muffins, as compared to the mango ice-cream?” Well, first, this is a hypothetical question and hence “mango muffins” shall gladly remain hypothetical. But, on a serious note, the moment you add a positive/negative tune to your statement, your respondents have higher chance to believe that than approach your question with a neutral standpoint. This bias can lead to invalidate your analysis later.


6. Be careful about your open-ended questions

Open Ended questions should be avoided as much as possible. Your surveys are not your interviews or focus groups that are meant to facilitate conversation. A survey is the only research method that holds the capabilities of both qualitative and quantitative research. We will discuss about analysis strategies in the part 3 of this mini-series but remember, more the number of open ended questions, more are chances of having scarce responses (not all respondents prefer to answer open ended questions) and higher are the chances of you not getting answer of the primary objective of your survey.


7. Choose your question types carefully

This blog post has established by now that your objective of doing a survey is the key. Because that determines what you ask. But just as important is what you ask, so is important how you ask your questions to your respondents. What that means is chose your question types appropriately. For example, only one answer is needed, use radio buttons. This point is doubly important because you should choose your question types from two standpoints – readability and analysis. Readability means your chosen question type should be easy to read both in web and mobile devices. That’s why should you have many answer options, using a drop-down radio is much better than pure radio button question. Similarly, any question type that is adding to scroll bar should be re-considered. Analysis, on the other hand, means your chosen question should have appropriate analysis method at the back end to extract an insight out of the question.


8. Make your answer choices well-rounded

Your answer choices reflect your depth in covering all aspects of your question. It is, therefore, a best practice to not just add the required options but also, in some form or shape, options like “all of the above”, “none of the above”, “others” etc.


9. Ensure your scales are set up correctly

Your analysis of the survey depends on how you have set up your questions, especially you scale-based questions. Make sure your scales have been given appropriate weights. As a general best practice, avoid using even-numbered scales because that removes a neutral ground. For example, the following odd-numbered scale is a preferred method:

- Very helpful (weight assigned: 5)

- Helpful (weight assigned: 4)

- Neither helpful nor unhelpful (weight assigned: 3)

- Unhelpful (weight assigned: 2)

- Very unhelpful (weight assigned: 1)


10. Ensure logical flow of your survey

Imagine going into an interview. Would you talk about your education, then childhood, then a little about your last job, followed by your education again? Hopefully not! In the same way, your survey is the representative of your thoughts to your respondents. Maintaining a logical flow not just ensures comfort of your respondents to understand your survey (i.e. you) but also build an unspoken form of trust that the party asking me questions knows what they are doing. Building this kind of trust, based on the quality of your survey, is the simplest form of marketing that you can do for your purpose. So, ensure that your surveys have a logical flow from one section to another and not just a bunch of random questions thrown in one single page.


Summary:

1. Understand the ‘why’ behind your survey

2. Ensure the length of your survey is not more than 5-7 mins

3. Keep the wording of your questions simple

4. Avoid asking double-barreled questions

5. Avoid leading and biased questions

6. Be careful about your open-ended questions

7. Make your answer choices well-rounded

8. Use matrices or grids carefully

9. Ensure your scales are set up correctly

10. Ensure logical flow of your survey


Stay tuned for the next post on learning what to do once you have your survey ready.


#survey #fundraising #donor #nonprofit #research #design #philanthropy