When you hear about a political survey, the results are often stated as “With a sample size of 200, 52% of the people prefer X to Y with a margin of error of 3%”, or some such. The figure “3%” is calculated based on the assumptions that:
- Everyone who is queried responded. I always ignore surveys, because they sound like spammers and telemarketers. This leads to bias, if members of one group always ignore polls.
- To be statistically accurate, the poll should start with the entire sample population, and pick names with a uniform distribution without replacement.
- Some pollsters may be lazy and use as a starting point the list of names from the previous poll, which leads to bias.
This can also leads to problems with an election. If the difference between candidates is small, the cost of time spent voting can be greater than the expected gain.