Sooner or later you will be called upon to design a survey and analyze the results. How you approach these can bias your results and hence your decisions.
In a series of videos, Dr. Nicola Petty of Dr. Nic’s Maths and Stats takes us through how to design a survey questionnaire and how to write good questions for it.
After that, we take a look at the survey analysis recommendations of Dr. John Schulz, principal teaching fellow within the Southampton Education School at Southampton University.
First: Designing a Questionnaire
There are five steps.
- Define the Problem
- What is the purpose?
- Who is the target population?
- What is the information going to be used for?
- What exactly do we want to find out?
- Plan how it will be administered
- Personal interview
- Written or online questionnaire
- Write the questionnaire
- Take each of the things you want to find out about and write a question for it
- Questionnaire structure is important
- Have a clear introduction
- Put interesting questions near the beginning
- Have a variety of question types
- Put demographic questions at the end
- Desk Check
- View the questionnaire as if seeing it for the first time
- Fill it out.
- Do the questions address what you wanted to find out?
- How will you record the responses?
- Pilot Survey
- Get a sample of people to fill it out
- Their responses will not be part of the survey
- “Watch” as they fill it out
- Anything confusing, boring, or annoying?
- Use what you find out to make changes in the questionnaire
Next: Writing Good Questions
A good questionnaire collects correct information and avoids bias or non-sampling error.
- Used when you want people to write what they like
- Also to avoid influencing their response
- They are difficult to analyze
- Offer the respondent a choice
- Keep questions as short as possible
- Use language appropriate to the subject and to the people you are asking
- Have only one idea per question
- Ask questions in positive terms
- In closed questions make sure choices do not overlap
You can view Dr. Petty’s videos here:
Finally: Analyzing a Questionnaire
Analysis depends on the type of questions in terms of level of measurement.
- Category type questions
- Ordinal type questions
- Continuous type questions
Steps in Analysis
- Code the questionnaire
- Allocate a number to each possible response
- Transfer the information to a spreadsheet or statistical analysis package
- Develop a strategy for analysis
- Summarize and describe the responses for each question
- Category Type Questions: Frequency of a response using percentages
- Ordinal Type Questions: Frequency of response using percentages
- Continuous Type Questions: Measures of central tendency: mean, median, mode
- Measures of dispersion: range, standard deviation
- Do a bivariate comparison of pairs of questions
- How do they interact or how are they different?
- Use a cross tabulation between pairs of category type questions
- To do a comparison of a category question with a continuous question using a comparison of means
- Use a scatter plot for two continuous questions
- Use inferential statistics to determine if results were achieved by chance or are significant
- For a single category type question, use a one-way Chi-Square test to see if one category is larger than another
- For comparing two category questions, use a two-way Chi-Square test
- To compare a category question with a continuous question, use a t-test if there are two groups in the category question or an ANOVA if there are more than two
- To compare two continuous questions, use correlation
You can view Dr. Schulz’s video here.
NOTE: Sources provided are to get you started. You can find a wealth of more detailed information online.