Designing a Questionnaire or Survey - statistics help

Summary

This video script outlines the essential steps for designing a high-quality questionnaire, using Helen's chocolate product survey as an example. It emphasizes defining the problem, choosing the survey method, crafting clear and respectful questions, conducting a desk check for clarity, and refining the questionnaire through a pilot survey. The process ensures the final questionnaire yields valuable data, highlighting the importance of following each step to avoid substandard results.

Takeaways

• 📝 The process of designing a questionnaire involves five key steps: defining the problem, planning the survey administration, writing the questionnaire, desk checking, and conducting a pilot survey.
• 🎯 Defining the problem is crucial and involves understanding the questionnaire's purpose, identifying the target population, determining the use of information, and identifying potential sources of variation.
• 👥 The target population should be carefully chosen based on the survey's goals, such as Helen focusing on new customers who buy chocolate but not her current products.
• 📊 The information gathered will guide product development, with Helen seeking to understand chocolate filling preferences and the preferences of high-spending chocolate consumers.
• 📝 The questionnaire should be structured to be interesting, easy to answer, and respectful, with demographic questions placed at the end.
• 🤔 The method of survey administration impacts the type of questions that can be asked, and Helen opts for an online survey for its ease and reach.
• ✍️ Writing the questionnaire requires careful construction of each question to avoid pitfalls and maintain a logical flow that engages respondents.
• 🔍 Desk checking involves reviewing the questionnaire for clarity, spelling, grammar, and ensuring it captures all necessary data points.
• 📊 A pilot survey is essential for identifying any confusion or issues in the questionnaire before full-scale implementation.
• 🔄 Feedback from the pilot survey should be used to refine the questionnaire, potentially leading to multiple rounds of desk checks and pilot surveys.
• 🌟 Following these steps leads to a high-quality questionnaire that will yield better data for decision-making.

Q & A

What are the five steps to designing a questionnaire as described in the script?

The five steps to designing a questionnaire are: 1) Define the problem, 2) Plan how the survey will be administered, 3) Write the questionnaire, 4) Desk Check, and 5) Pilot Survey.

Why is it important to define the problem when designing a questionnaire?

Defining the problem is crucial because it helps to clarify the purpose of the questionnaire, identify the target population, determine the use of the information, and understand what exactly needs to be found out.

What is Helen's purpose for designing a new questionnaire?

Helen's purpose is to introduce a new chocolate product and find out what type of chocolate product is most likely to be successful in the market, particularly focusing on attracting new customers who buy chocolate but do not currently buy Chocconutties.

Why is it necessary to consider the target population when designing a questionnaire?

Considering the target population ensures that the questions are relevant to the group that the survey aims to gather information from, which in Helen's case, are people who buy chocolate but are not current Chocconutties customers.

What method of survey administration did Helen choose and why?

Helen chose an online survey method because it fits her budget, allows her to reach more than just current customers, and makes creating the questionnaire and analysis much easier.

What are some of the potential pitfalls in writing questions for a questionnaire?

Potential pitfalls in writing questions include unclear language, leading questions, double-barreled questions, and questions that may be considered sensitive without proper ethical approval.

Why is the structure of a questionnaire important?

The structure of a questionnaire is important because it should be interesting, easy to answer, and respectful. A well-organized questionnaire with a clear introduction and a variety of question types helps maintain the respondent's interest and ensures they continue to answer.

What should be included in the introduction of a questionnaire?

The introduction should explain why the data is being collected, for whom it is intended, and assure respondents about the confidentiality of their responses.

What is the purpose of a Desk Check in the questionnaire design process?

The purpose of a Desk Check is to review the questionnaire as if seeing it for the first time, ensuring clarity, correctness of spelling and grammar, and verifying that all necessary information is being collected. It also helps in deciding how to record responses and checking the ease of coding.

What insights can be gained from conducting a Pilot Survey?

A Pilot Survey provides insights into any confusion or issues respondents may have with the questionnaire, helps identify questions that may need alternative responses, and allows for adjustments to be made before the actual survey.

How does the script suggest ensuring the ethicality of a questionnaire?

The script suggests ensuring ethicality by avoiding sensitive questions without proper approval, being respectful of respondents' time by including only relevant questions, and expressing gratitude at the end of the questionnaire.

Outlines

00:00

📝 Questionnaire Design Essentials

This paragraph outlines the five critical steps in designing a questionnaire: defining the problem, planning the survey administration, writing the questionnaire, conducting a desk check, and a pilot survey. Skipping steps four and five can lead to substandard questionnaires and poor data. Helen's case of selling Choconutties is used as an example to illustrate these steps. The purpose of the questionnaire is to determine the most successful type of chocolate product to attract new customers, with a focus on popular fillings and preferences among high-spending chocolate consumers. The target population is non-customers of Choconutties who buy chocolate, and the questionnaire will explore potential variations in preferences between genders. The chosen method of survey administration is online, which is cost-effective and reaches a broader audience. The paragraph emphasizes the importance of a well-structured and ethical questionnaire, including clear instructions, interesting content, and respect for respondents' time.

05:01

🔍 Refining the Questionnaire Process

The second paragraph discusses the importance of refining the questionnaire through a desk check and a pilot survey. The desk check involves reviewing the questionnaire for clarity, grammatical correctness, and relevance to the research objectives. It also includes checking the ease of data recording and coding. The pilot survey is a trial run with a small group of respondents to identify any confusion or issues in the questionnaire. Feedback from the pilot survey is used to make necessary adjustments. If significant changes are made, another desk check and possibly another pilot survey are recommended. The paragraph concludes by emphasizing the goal of creating a high-quality questionnaire that will yield better data, and it credits the video to the Statistics Learning Centre, which offers additional resources for learning.

Keywords

💡Questionnaire

A questionnaire is a research instrument consisting of a series of questions for the collection of information from respondents. In the video, the questionnaire is central to the process of gathering data to help Helen, the seller of Choconutties, understand market preferences for a new chocolate product. The script emphasizes the importance of designing a questionnaire that is well-structured and ethically sound to ensure the quality of the data collected.

💡Survey Administration

Survey Administration refers to the method by which a survey is conducted, such as through phone, personal interview, written, or online questionnaires. The script discusses the selection of an online survey method for Helen, considering her budget and the need to reach a broader audience beyond her current customer base. This choice impacts the type of questions that can be asked and the ease of creating and analyzing the questionnaire.

💡Desk Check

Desk Check is a preliminary review of a questionnaire to identify any potential issues before it is administered to respondents. The script describes this step as a critical process where the questionnaire is examined for clarity, spelling, grammar, and relevance of questions. It ensures that the questions align with the research objectives and are easy to understand and answer.

💡Pilot Survey

A Pilot Survey is a small-scale version of the main survey conducted to test the questionnaire's effectiveness and identify areas for improvement. The script mentions observing respondents as they complete the questionnaire, noting any confusion or issues, and using this feedback to refine the questionnaire before the full-scale survey.

💡Target Population

The target population in a survey is the group of individuals that the researcher is interested in studying. In the context of the video, Helen's target population is non-customers of Choconutties who purchase chocolate, as she aims to attract new customers and understand their preferences for her new product.

💡Data Collection

Data Collection is the process of gathering information from respondents through a survey. The script highlights the importance of defining the purpose of the questionnaire to guide the data collection process, ensuring that the information gathered will be used effectively to develop new chocolate products.

💡Questionnaire Structure

Questionnaire Structure refers to the organization of questions within a survey. The script emphasizes the need for a logical flow, starting with interesting questions to capture attention and ending with demographic questions. It also mentions the importance of clear instructions and grouping questions by type or topic for ease of response.

💡Ethical Considerations

Ethical Considerations in a questionnaire involve ensuring respect for respondents' privacy and avoiding sensitive questions without proper approval. The script points out the need for ethical approval and the importance of being respectful of respondents' time by including only relevant questions and maintaining confidentiality.

💡Confidentiality

Confidentiality in the context of a questionnaire means ensuring that the responses of the participants are kept private and are not associated with their identities. The script advises explaining the confidentiality of responses to the respondents to build trust and encourage honest participation.

💡Data Analysis

Data Analysis is the process of examining and interpreting the data collected through a survey. Although not explicitly detailed in the script, the process of analyzing the pilot survey responses is mentioned, which is crucial for identifying trends, patterns, or areas of confusion that may require changes to the questionnaire.

💡Variation

Variation in a survey context refers to the differences in responses or preferences among different groups of respondents. The script uses the example of potential variation in chocolate preferences between men and women, indicating the need to design the questionnaire to capture these differences and tailor the product to each market segment.

Highlights

There are five key steps to designing a high-quality questionnaire.

Defining the problem is crucial for the questionnaire's purpose and target audience identification.

Planning the survey administration method impacts the types of questions that can be asked.

Writing the questionnaire requires careful structuring to maintain interest and respect for respondents' time.

A desk check helps identify potential confusion and ensures all necessary information is covered.

A pilot survey is essential for observing real-time responses and making necessary adjustments.

Skipping steps four and five can lead to substandard questionnaires and poor data.

Helen's example illustrates the questionnaire design process for developing a new chocolate product.

The target population for Helen's survey is non-Chocconutties customers who buy chocolate.

Helen is interested in chocolate fillings popularity and preferences among high-spending consumers.

Variation in chocolate preferences between genders is a key consideration for Helen's market segmentation.

Online surveys are chosen for their ease of creation, administration, and analysis.

Questionnaire introduction should explain the purpose, beneficiaries, and confidentiality.

Ethical considerations are vital, avoiding sensitive questions without proper approval.

Demographic questions should be placed at the end of the questionnaire.

Clear instructions and grouping of questions enhance the ease of answering.

Recording responses in a structured table or spreadsheet is important for data analysis.

If significant changes are made after the pilot survey, a new desk check may be necessary.

The video is provided by the Statistics Learning Centre, offering resources for learning.

Transcripts

Designing a Questionnaire.

There are five steps to designing a questionnaire:

Define the problem.

Plan how the survey will be administered.

Write the questionnaire.

Desk Check.

Pilot Survey.

It can be tempting to leap straight to step three,

and also to miss out steps four and five,

but this will result in a substandard questionnaire and poor data.

Helen sells Choconutties.

She has asked us to help her design a questionnaire

to help her develop a new product.

We will use this as an example to illustrate the steps.

Step 1) Define the problem.

We can do this by answering these questions:

What is the purpose of the questionnaire?

This is important whether we are working for someone else or for ourselves.

Who is the target population?

In other words, who would we like to ask if we could ask everyone who matters?

What is the information going to be used for?

What exactly do you want to find out?

Think about what are possible sources of variation.

The purpose statement for Helen is:

Helen is planning to introduce a new chocolate product.

She wants to find out what type of chocolate product

is most likely to be successful in the market.

Helen is most interested in attracting new customers,

so the target population is people who don't already buy Chocconutties

but do buy chocolate.

The information will be used to help Helen develop new chocolate products.

Helen wants to know, specifically, what chocolate fillings are popular,

and which kind of chocolate is preferred by people who spend a lot on chocolate.

Helen thinks there may be variation

in chocolate preference between men and women,

and wishes to design specifically to each market,

so will need to ask if the respondent is male or female.

Step 2) Plan how the survey will be administered.

We can choose between phone, personal interview, written

or online questionnaire.

These methods each have different advantages and disadvantages.

The method of administration affects the types of questions

that can be asked.

For her budget,

and because Helen wants to reach more than just current customers,

Helen wants to use an online survey method.

This will make creating the questionnaire and doing the analysis much easier.

Step 3) Write the questionnaire.

Take each of the things you want to find out about

and write a question for it.

There are many potential pitfalls in writing questions.

These are covered in another video, Writing Good Questions.

Questionnaire structure is important.

A good questionnaire should be interesting,

easy to answer and respectful.

An informative title, clear introduction and interesting, well-organised questions

help the respondent to keep answering.

The introduction needs to explain

why the data is being collected and for whom.

This should also explain about confidentiality of the responses.

Interesting questions near the beginning

help capture the attention of the respondent,

and a variety of question types can help maintain interest.

The demographic questions, like age, income and gender

should be left to the end.

Clear instructions and sensible grouping of questions by type or topic

make the questionnaire easier to answer.

A questionnaire must be ethical.

You must be careful not to ask questions on sensitive subjects

without getting proper ethical approval or advice.

Be respectful of the person's time by only including relevant questions,

and say, "Thank you", at the end.

Step 4) Desk Check.

Look at your questionnaire as if you were seeing it for the first time.

Read the instructions, and fill out the answers.

See what could be confusing.

Make sure your spelling and grammar are correct.

Go through the list of things you wanted to find out

and put ticks next to the questions that provide that data.

Look closely at any questions with no tick

and decide if they need to be in the questionnaire.

Consider how you would record the responses.

Generally, you make a table or spreadsheet with a row for each person

and a column for each question.

Check to see that your questionnaire is easy to code, like this.

Because Helen is using an online survey provider,

much of this will be done for us.

Now use what you found out from your Desk Check

to correct your questionnaire, ready for Step 5) Pilot Survey.

This is where you get some people to answer your questionnaire.

These responses will not form part of the actual survey.

Watch while they fill out the questionnaire.

Note anything that seems confusing to them.

Look at what the people have written.

Ask them if they found any of it confusing, boring or annoying.

Thank them.

If you've done a large Pilot Survey, you could do a quick analysis of the data.

See if there are any questions that several people answer "Other" to.

This might mean that you need to add

another response alternative in that question.

Use what you found out from the Pilot Survey

to make changes to your questionnaire.

If you make lots of changes, do another Desk Check

and, possibly, another Pilot Survey.

You want your questionnaire to be good.

If you follow these steps, you should end up

with a high quality questionnaire ready for administering,

which will result in better data.

This video was brought to you by Statistics Learning Centre.

Visit our website for more resources to help you learn.