Both my regular readers may have noticed that my scorn and outrage are mostly confined to issues of bad science (1), by which I mean capital ‘s’ Science! (gotta say it like the dude on the Thomas Dolby record) – the kind of quasi-religious gee-whiz science championed by Sagan and reported on by a fawning, power-worshiping press almost completely ignorant of what they’re talking about.
See, my reaction is of a lover hearing his beloved misrepresented and trampled upon. I’ve always loved figuring stuff out – just a personal quirk. The first book I recall studying was the gigantic Webster’s we had at home – but only the Our Universe (or some such) section at the front, with pictures of solar system and various astronomy facts. Just thought it was way cool. I tried to build a boat when I was 5. I did build a retractable ladder for my clubhouse at 11, where a pulley system allowed you to raise or lower the ladder from the ceiling. Stuff like that.
When I discovered those Time-Life science books in 4th grade, I was in heaven – all sorts of stuff figured out, and little projects you could do! This was a world I belonged in!
Soon, my reading revealed the Scientific Method, as a sort of negative – all these examples of how people had gotten it wrong, and how other people had straightened it out. I wasn’t sophisticated enough then (and I’m not sure I am now) to formulate a positive description of the Scientific Method that can stand up to a comparison of what real scientists really do, but I can spot the phoneys.
In my head, the Millikan oil drop experiment is a sort of icon for real science. A set of well-defined assumptions, a well controlled apparatus, clean math, well understood and verifiable observations, ending in a protocol that anybody can theoretically reproduce. I’ve done it. Thousands have done it. It’s cool. It works! (2)
So, I tend to compare, at least unconsciously, claims that Science! Has Shown with the oil drop and other classic experiments, and see how they looks. If the assumptions are not clearly presented and well understood, or the apparatuses are not all well understood and controlled, or the math – at least, insofar as I can understand it – is murky, or the observations are ill-defined or subjective, or the protocol is difficult or impossible to reproduce – well, one or more of these problems may happen despite honest effort, but the claims made should be similarly toned down, and the problems acknowledged. If lots of these issues are evident, maybe silence is the best policy.
We are deluged with bad science. This comes, it seems to me, from two causes, both of which trigger in me a visceral fury:
1. Egomania and Ambition, or the ‘Wouldn’t the Aura of Science Kick My Subjective Fantasies Up a Notch?’ cause. Freud is the poster child: the bulk of his claims are not subject to any test whatsoever even in theory – you can’t even in theory come up with a way to measure or even observe what he claims to have observed (3). He earns my particular ire for his sole reliance on Bulverism to refute his critics – they only point out the utter silliness of Freud’s claims because they are sexually repressed, you see. There isn’t any other argument, unless you count ‘because I say so, and I’m Sigmund Freakin’ Freud!’ as an argument. (And yes, I’ve read a bunch of Freud. And Jung.)
But Freud is only an especially egregious example. Pretty much all of psychiatry and psychology and almost all of sociology are covered here. The test is: could you objectively know what you claim to know even in theory? For example, the mental structure and motivations of another person – how, scientifically, would you know that? A lesser set of offenders would be, for example, phrenology and its modern heirs, brain-imaging studies, where the quality of the observations hardly warrants any scientific claims at all – but at least in theory, some of the claims could be validated by experiment. In these cases, we look at the kind and extent of the experiments upon which the claims are made, and the nature of the claims themselves: You can’t claim that a particular bump on a skull or a particular area of the brain lighting up tells you anything much about how a person feels or thinks, without tons of data and lots of controls. And even then, it’s iffy.
2. Money, and its algebraic equivalents, Power and Fame. Read Eisenhower’s speech? To sum up: once government starts paying the scientific piper, government calls the tunes. Not only in terms of what gets studied, but in terms of what gets ‘discovered’. Shockingly, a huge number of government studies conclude we need more government action. Imagine. And I’ve yet to hear of one that concludes that any area at all could benefit from less government.
Sure, industries have money and fund studies, too, and we should be rightly suspicious of their conclusions – but their efforts are dwarfed by government spending in its various guises.
Now, these two causes are hardly mutually exclusive, which is how you get studies showing that racism is already established in 9 month old babies. H/T to Mike Flynn on this. William Briggs nicely dissects it. This study is what we plain-speaking primitives call a ‘fraud’, even allowing for the author’s (4) disavowing the places where the journalists promptly took it, because the number of babies in this study – single digits – warrants no conclusions at all. Which brings us to:
Then, there’s how science is reported. Oh. My. Goodness! The sad part is that there are a few – not many – science reporters who have some science background, but these are almost without exception of the Sagan ‘Bow, You Peons!’ school of science education.
Key indicators of Science! reporting as opposed to real attempts to understand the physical world:
– Uses the words ‘believe’ or ‘consensus’;
– presents survey data as science. At best, survey data might be an input item, a data point. It’s never science in any but a highly constrained sense;
– bemoans how the ignorant, unwashed masses just don’t get it;
– claims that the science proves beyond any educated doubt that the government needs to do something right now or we’re all doomed;
And this is apart from the major characteristic: getting the science wrong, most commonly by restating some highly conditional preliminary suspicions as stone-certain conclusions.
And let’s not even talk about people of bad intent, or people getting professionally cowed or having their funding threatened, or simple out and out lies.
At least, not today.
1. OK, I can get worked up over education. And liturgical music. and probably some other stuff, too.
2. Even the existence of questions about Millikan’s too good to be true observation prove the point: the experiment is so clean and convincing and so readily reproducible that people can even notice something as subtle as Millikan’s unconscious data collection bias.
3, An interesting exception is Freud’s discovery of the subconscious – in this one case, Freud is making a philosophical, even metaphysical, argument: that a certain class of observations would not be possible if a subconscious did not exist. This may or may not be true, but regardless, the argument is philosophical, not scientific.
4. The author mostly worked on getting government funding for scientific studies – go figure.