7.4. Questionnaire design for minimisation of measurement error and ease of analysis
It is important to word questions carefully, using neutral language, clearly and unambiguously to avoid misunderstanding and consequent errors in data supplied. Carrying out a pilot survey of a questionnaire (see section 7.7), or part of it, involving any new questions is essential in order to check that the questions are appropriately worded. Modifications of existing questions are best tested in the same way, unless the change is minor.
It is worth considering the order of questions asked. The usual guideline is to ask more general questions before more specific ones. This avoids attitudes/responses from becoming fixed early on and may encourage a more flexible way of thinking. It also allows for appropriate question routing, i.e. based on their response to certain questions the respondent is then directed to go to the next appropriate question for them to answer. For example, the questionnaire can state something like "If you responded "yes" to this question, please go next to question X". This is important in surveys drawn randomly from membership lists of beekeeping associations, where not all members may be active beekeepers at the time of the survey and it cannot be determined prior to carrying out the survey which members are active beekeepers. Asking whether or not the respondent is an active beekeeper early on in the survey allows asking such respondents any questions directed at them specifically, while directing the active beekeepers to the start of the main part of the questionnaire and the questions intended for them.
For ease of data coding and analysis it is best to use closed format questions where possible, with a fixed number of response options of most interest and/or thought to be the most common, and to provide an "other" or "not applicable" category to cater for responses that cannot be fitted into the supplied list of responses, with the means to provide further details of the answer to the question. Closed questions with a given format make it possible to compare responses from surveys of different populations, for example in different countries, or of the same or a similar population at different times. Completely open questions inviting a written response are much less easy to deal with in data coding and analysis and are best avoided. The number of response options is best not to be too long, to avoid confusion or error.
Questions asking for a numerical response are best worded and set out to allow the respondent to supply the exact number, of colonies managed, for example, as the answers can be categorised later if required but having the exact numbers provides more information for analysis.
It is well-known that asking sensitive questions in surveys (see section 7.3.3.) is less likely to elicit an accurate or complete response than less emotive questions. Some survey methods are more successful in this matter than others (Schaeffer et al., 1990).
Questions requiring more knowledge than a participant has are likely to be answered inaccurately. Either some background information should be provided, or a screening question(s) should be asked first to determine whether or not it is appropriate for the participant to be asked the particular question of interest.
While constructing the questionnaire and accompanying documentation, including a covering letter/invitation to participate and instructions to the survey participant, a coding sheet should also be prepared. In online surveys, responses will automatically be entered into a database, and the coding of them is part of the questionnaire design (0=no, 1=yes, N/A for missing, for example). In surveys requiring manual data entry, a coding sheet is important for consistent translation of survey responses into data entered in a spreadsheet. This is especially important in situations where there is more than one person involved in data entry.