science

Participation bias

Participation bias is a science topic covered in the lgStudy science library. This page brings together a partial reference excerpt, illustrations, worked examples, real-world applications and a short study plan, so you can understand Participation bias rather than just read about it. In short: Participation bias or non-response bias is a phenomenon in which the results of studies, polls, etc. become non-representative because the participants disproportionately possess certain traits which affect the outcome. These traits mean the sample is systematically different from the target population, potentially resulting in biased estimates.

Key takeaways

Participation bias belongs to science; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect Participation bias to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of Participation bias from memory before moving on to harder problems.

Reference excerpt

Participation bias or non-response bias is a phenomenon in which the results of studies, polls, etc. become non-representative because the participants disproportionately possess certain traits which affect the outcome. These traits mean the sample is systematically different from the target population, potentially resulting in biased estimates. For instance, a study found that those who refused to answer a survey on AIDS tended to be "older, attend church more often, are less likely to believe in the confidentiality of surveys, and have lower sexual self disclosure." It may occur due to several factors as outlined in Deming (1990). Non-response bias can be a problem in longitudinal research due to attrition during the study.

Example If one selects a sample of 1000 managers in a field and polls them about their workload, the managers with a high workload may not answer the survey because they do not have enough time to answer it, and/or those with a low workload may decline to respond for fear that their supervisors or colleagues will perceive them as surplus employees (either immediately, if the survey is non-anonymous, or in the future, should their anonymity be compromised). Therefore, non-response bias may make the measured value for the workload too low, too high, or, if the effects of the above biases happen to offset each other, "right for the wrong reasons." For a simple example of this effect, consider a survey that includes, "Agree or disagree: I have enough time in my day to complete a survey." Prior to the 1936 U.S. presidential election, The Literary Digest mailed out 10 million questionnaires, of which 2.38 million were returned. Based on these returns, they predicted that Republican candidate Alf Landon would be elected president with 57.08% of the popular vote and 370 of 531 electoral votes: in the election, Landon received only 37.54% of the popular vote and eight electoral votes in an unprecedented landslide loss to Democratic incumbent President Franklin D. Roosevelt. Subsequent research published in 1976 and 1988 concluded that non-response bias was the primary source of this error, although their sampling frame was also quite different from the vast majority of voters. Non-responders have been shown to be associated with younger patients, poorer communities and those who are less satisfied and subsequently could be a source of bias.

Test

There are different ways to test for non-response bias. A common technique involves comparing the first and fourth quartiles of responses for differences in demographics and key constructs. In e-mail surveys some values are already known from all potential participants (e.g. age, branch of the firm, ...) and can be compared to the values that prevail in the subgroup of those who answered. If there is no significant difference this is an indicator that there might be no non-response bias. In e-mail surveys those who didn't answer can also systematically be phoned and a small number of survey questions can be asked. If their answers don't differ significantly from those who answered the survey, there might be no non-response bias. This technique is sometimes called non-response follow-up.

Response rate A common assumption is that response rates are linked to non-response bias, with lower response rates incurring a greater likelihood for results to be influenced by non-response bias. Academic research has disputed substantial linkages between response rate and non-response bias. A meta-analysis of 30 methodological studies on non-response bias by Robert M. Groves found that the coefficient of determination for variance in non-response bias by response rate was only 0.11, making it a weak predictor of non-response bias. Another meta-analysis of 44 methodological studies similarly found that methods that resulted in higher response rates, such as prior notification and incentives, would not necessarily reduce non-response bias and could sometimes even show increased non-response bias. Some academic journals, particularly in the medical space, require minimum response rates to publish survey research as a means of mitigating non-response bias. For instance, JAMA has implemented a requirement of a 60% response rate for survey research, which has been compared to other heuristics like 0.05 p-values for statistical significance in social science research. The pursuit of higher response rates can be counterproductive due to its questionable relationship with non-response bias and potentially unnecessary costs spent on methods of boosting response rates that could be better applied elsewhere. It may also result in gatekeeping of surveys that may be valid on their merits, but fail to satisfy a heuristic requirement on response rates.

Related terminology Self-selection bias is a type of bias in which individuals voluntarily select themselves into a group, thereby potentially biasing the response of that group. Response bias is not the opposite of non-response bias, but instead relates to a possible tendency of respondents to give inaccurate or untruthful answers for various reasons.

Worked examples

Example 1 — a first encounter with Participation bias

Start with the simplest possible case. Write down what Participation bias claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to Participation bias before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about Participation bias ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of Participation bias

In research: Participation bias appears in science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses Participation bias in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: Participation bias is common in secondary-school and first-year university syllabi. It links to neighbouring topics Experimental bias, Social research, Survey methodology, so understanding it makes those chapters shorter.
In everyday life: Look for Participation bias outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

Ask Teacher Smith questions about this articleOpens your AI tutor with a question about “Participation bias” →

Affiliate

Preply — study more efficiently by working with a personal tutor. 50% off.

How to study Participation bias in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what Participation bias means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain Participation bias out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is Participation bias in simple terms?

Why does Participation bias matter?

Because it connects several science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study Participation bias?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on Participation bias.