Design 

Sooner or later you will be called upon to design a survey and analyze the results. How you approach these can bias your results and hence your decisions.

In a series of videos, Dr. Nicola Petty of Dr. Nic’s Maths and Stats takes us through how to design a survey questionnaire and how to write good questions for it. 

After that, we take a look at the survey analysis recommendations of Dr. John Schulz, principal teaching fellow within the Southampton Education School at Southampton University.

First: Designing a Questionnaire

There are five steps.

  1. Define the Problem
    1. What is the purpose?
    2. Who is the target population?
    3. What is the information going to be used for?
    4. What exactly do we want to find out?
  1. Plan how it will be administered
    1. Phone
    2. Personal interview
    3. Written or online questionnaire
  1. Write the questionnaire
    1. Take each of the things you want to find out about and write a question for it
    2. Questionnaire structure is important
    3. Have a clear introduction
    4. Put interesting questions near the beginning
    5. Have a variety of question types
    6. Put demographic questions at the end
  1. Desk Check
    1. View the questionnaire as if seeing it for the first time
    2. Fill it out.
    3. Do the questions address what you wanted to find out?
    4. How will you record the responses?
  1. Pilot Survey
    1. Get a sample of people to fill it out
    2. Their responses will not be part of the survey
    3. “Watch” as they fill it out
    4. Anything confusing, boring, or annoying?
    5. Use what you find out to make changes in the questionnaire

Next: Writing Good Questions

A good questionnaire collects correct information and avoids bias or non-sampling error.

Open Questions

  • Used when you want people to write what they like
  • Also to avoid influencing their response
  • They are difficult to analyze

Closed Questions

  • Offer the respondent a choice

Some Guidelines

  • Keep questions as short as possible
  • Use language appropriate to the subject and to the people you are asking
  • Have only one idea per question
  • Ask questions in positive terms
  • In closed questions make sure choices do not overlap

You can view Dr. Petty’s videos here:

Design

Writing Questions

Finally: Analyzing a Questionnaire

Analysis depends on the type of questions in terms of level of measurement.

  • Category type questions
  • Ordinal type questions
  • Continuous type questions

Steps in Analysis

  1. Code the questionnaire
    1. Allocate a number to each possible response
  2. Transfer the information to a spreadsheet or statistical analysis package
  3. Develop a strategy for analysis
  4. Summarize and describe the responses for each question
    1. Category Type Questions: Frequency of a response using percentages
    2. Ordinal Type Questions: Frequency of response using percentages
    3. Continuous Type Questions: Measures of central tendency: mean, median, mode
      1. Measures of dispersion: range, standard deviation
  5. Do a bivariate comparison of pairs of questions
    1. How do they interact or how are they different?
    2. Use a cross tabulation between pairs of category type questions
    3. To do a comparison of a category question with a continuous question using a comparison of means
    4. Use a scatter plot for two continuous questions
  6. Use inferential statistics to determine if results were achieved by chance or are significant
    1. For a single category type question, use a one-way Chi-Square test to see if one category is larger than another
    2. For comparing two category questions, use a two-way Chi-Square test
    3. To compare a category question with a continuous question, use a t-test if there are two groups in the category question or an ANOVA if there are more than two
    4. To compare two continuous questions, use correlation

You can view Dr. Schulz’s video here.

NOTE: Sources provided are to get you started. You can find a wealth of more detailed information online.