Another Thought on Polls

Two related problems with polls: Polls are not science. Polls do not tell you what people are thinking, only what they are willing to say. Therefore: two easy rules a rational person should follow: do not use polls in lieu of real science. Do not imagine polls tell you about what people think.

On top of this, polls are notoriously easy to manipulate, in the following well-understood ways: how you frame the question, both in the nature of the question itself and in what questions or statements precede it, has a great influence on what answer you get; how you select the people polled – for example, polling people over the phone selects only people with phones and the time to spend answering questions over them; polling college students selects for college students with the time and inclination to take polls. In general, all the answers you get will be from people who care to give answers – and it’s a leap to imagine that’s representative of people in general; and finally, any time a poll is interpreted to mean anything other than ‘this is what these people were willing to say in response to these questions asked in this way at this time’ the research is being spun.

Actually, I got a little ahead of myself there: that last one is not well understood by very many people, so let’s go into it a bit. I get my poll results back from the dude who minored in math who compiled them into a nice table. This table presents both raw numbers and percentages. It can be fairly assumed that the arrangement of the data reflects the reason the poll was done – all polls are done for a reason and with a theory in mind. This point should be obvious: pollsters don’t wander about asking random people random questions about whatever moves them at the moment – they ask specific questions of specific people in order to (in the best, honest case) gather information about the opinions those people are willing to share about the topics of interest. (On this date, in this place, under these conditions – it matters, say, if you’re polling about healthcare or police behavior in a park on the Forth of July or in an emergency room after a riot.)

Given all that, if a poll is being said to give causal relationships, or even insight into how people think, it is reaching far beyond what *any* polling results can give you. In the last post, we had an egregious but sadly not uncommon example: the poll asks a question* along the lines of “the decision to vaccinate children should rest with the parents – yes or no?”

Some people answered ‘yes’. Some people separately answered another (unstated in the report) question: “Democrat, Republican, Independent, Other?” as Republican. There was some overlap between those that answered “Yes” to the first question and “Republican” to the second – as there was with every other possible combination of answers.

So? You could say, maybe, that, based on the poll results, self-identifying as a Republican correlates with a higher incidence of “Yes” answers when compared to Democrats and Independents. That’s really about all one can say. But to speculate that whatever causes one to self-identify as a Republican to a pollster also causes one to answer “yes” is magical mind-reading. You just don’t know – that’s not something poll results can tell you even in theory. To go further, and assert that what causes one to be a Republican is the same anti-science ignorance that makes one more likely to oppose vaccination is beyond ridiculous. Not only does polling in itself not provide such information, the actual questions asked (as far as we can tell) weren’t even about those issues – one could completely support vaccinations AND think parents should decide – those are not remotely mutually exclusive positions.

All such attempts at interpretation show is whatever the users of the polling results wanted to say anyway.

So, the rule is: you see somebody using poll results to show that their enemies are stupid? They are merely showing that they themselves are too stupid to use poll data correctly.

Here’s an example of polling that makes some sense, for comparison: based on experience, I can assemble a group of subjects to poll who are likely to vote. I ask them: are you likely to vote next election? If so, do you favor candidate A or B? Then, I can reasonable conclude: X% of people who say that they are likely to vote next election say they favor candidate A. Such information might be very useful for planning a political campaign.

But that’s about it. But because elections are winner-take-all fights to the death, political pollsters while away the hours trying to tease out the thought processes of voters so as to design more successful campaigns. One thing that has proven irresistible is using those guesses about how people think as political tools. The Left loves propagating the idea that they are smart and enlightened, while their enemies are stupid and benighted – because it works with their base. It also has made some inroads into the historical stronghold of Republicans – college graduates. After over half a century where a college degree strongly correlated to being a Republican, these days, it’s only a strong predictor for men. Women with college degrees, who are (as it relentlessly reported) less likely to take those nasty hard science, math, and engineering classes and more likely to take Studies degrees, have started to lean a little left.

I will merely note that this trend parallels the downward trend in actual academics: back when high school students learned Greek and calculus, college grads were reliably Republicans; now that up to 50% of incoming freshmen at our elite colleges must take remedial math and English, this relationship has weakened. Based on the actual evidence, one might be forced to consider the possibility that, perhaps, intelligence doesn’t incline one to become a liberal Democrat. But that’s a thought requiring a degree of subtlety and open-mindedness completely out of place when there are elections to be won.

* Any reporting on a poll that does not include the actual questions asked in the exact form they were asked is committing fraud. Full stop. It is spinning results and attempting to manipulate the reader. In other words, it is impossible for a reasonable reader to conclude anything at all about poll results without knowing the specific questions. Even then, very little can be concluded – but without the real questions, the reader’s analysis is over before it began.


Author: Joseph Moore

Enough with the smarty-pants Dante quote. Just some opinionated blogger dude.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s