When we last left off, we were discussing claims and evidence. Now let’s talk about the quality of claims, evidence, and the relationship of claims and evidence. This it probably Part 1 – topic spiral potential: high.
To cut to the chase: a reasonable, useful claim is specific, expressed in unambiguous terms, and subject to logical and real-world contradiction.
Thus, reasonable, useful evidence addresses specific claims, according to the rules of logic.
This all may seem pedantic nonsense. If so, you will find real science is all about pedantic nonsense. Ask a scientist a simple question: what is the boiling point of water? and, if he is answering as a scientist, you will get:
- discussions about what a state change is, energy thresholds, margins of error, and limits of observation;
- A laundry list of conditions that affect the observed boiling point of water: air pressure, purity of the water;
- THEN he might say: 100C, GIVEN all the definitions, conditions, and caveats listed above.
The boiling point of water is about as simple a scientific question as one can ask. See Millikan’s classic oil drop experiment (my favorite, and the fanciest experiment I’ve every personally done) for something a little more complicated. To calculate the charge of an electron, Millikan and Fletcher had to define, develop and measure a whole bunch of things, e.g., the size and mass of aerosol oil droplets and the viscosity of air (which changes with temperature and pressure). They needed to design and build a device that 1) created tiny oil droplets; 2) generated electrons in such a way that some of them would stick to those oil droplets; 3) provided a consistent, measurable way to observe the oil droplets thus created; 4) had a magnetic field of known strength that they could turn on and off at will. THEN you spend hundreds of hours (in addition to the hundreds you spent coming up with the experimental concepts and building and perfecting the device) risking blindness to gather thousands of observations.
Millikan did all that, and a ton of math, then got to say that the charge of the electron is 1.5924(17)×10−19 C (1) and collect his Nobel Prize.
In the real world, few people understand the question: what is the charge of an electron? let alone feel any need to know the answer.
A scientific claim is a claim that answers a scientific question. (I’m a regular Obvious Oscar today!) If the question itself does not go through the refining and defining required to hammer it into scientific shape, cleaning up as much as possible all ambiguities and and establishing the limits and conditions, then the pseudo-scientific claim that science has answered such a question is, and must be, wrong.
The above is a round-about way of addressing the nature of science as discussed here for a decade or more. Science, as John C. Wright points out often, is the study of the metrical properties of physical objects. If the question does not concern the measurement of the properties of something you can see, hold in your hand, smell, taste, hear – then it’s not a scientific question. Note: this does not mean your question is unimportant or wrong, merely that you’re not going to be able to use science to answer it. Most of life’s really important questions – should I ask her to marry me? what is the right thing to do? how should I spend my life? and so on – are not science questions. We have to come up with other ways to answer them.
It should be clear at this point that scientific evidence must be weighed by how well, if at all, it addresses a well formed scientific question. Badly formed or categorically wrong questions cannot, as in, CANNOT be answered scientifically. Science will not tell me if I should ask this particular woman to marry me; science has nothing to say about the proper course of action for any human decisions. Getting an ought from an is is difficult, if, indeed, the impossible can be called difficult. (That’s an Aristotle joke, there.)
Thus, people are making a categorical error when they claim to be ‘following the science’ when the do or promote actions. There’s always more to the question. Ex: if someone is injected with these chemicals, they will die. Therefore, IF we don’t want a particular someone to die, we should not inject them with these chemicals. So, do we want them to die? Are they an innocent child, or a serial killer of innocent children on death row? Could have different answers. IF the person OUGHT to die is not a question science can answer.
To be convincing or even relevant scientifically, evidence requires a good, clean scientific question to be run up against. Take an example I’ve used before: I saw a report once that a certain migratory butterfly population had decreased 87.3% (say. Numbers are for illustration only.) The obvious scientific question this ‘evidence’ would be addressing is: are there fewer butterflies of this particular type in a specified time period as opposed to another specified time period? Simply putting the question in that format should suggest the conditions and definitions needed in order to evaluate evidence, if any:
- How is the counting of butterflies being done?
- How is the accuracy of the counting assured? (This means, in Feynman’s classic formulation, that the makers of the claim/presenters of the evidence are required by the honesty implicit in the pursuit of science to list any possible ways they can think of that their conclusions, methods, or data could be wrong.)
In turn, these questions intended to clear up and make scientific the more general question do, themselves, raise questions. And here’s the point of this exercise: if the study or report authors or claimants cannot show that they have done the thought-smithing needed to define and clarify the question they claim to be addressing, then, put bluntly, it’s not science. In the above example, at least, the ‘researchers’ would need to assure us that:
- Where they are looking for the butterflies is where the butterflies are – namely, that they didn’t simply take another route, or take the usual route at a different time. In other words, that their count is in fact a count that includes all the relevant butterflies.
- How they counted those thousands of butterflies is at all accurate.
And so on.
In the real world, the speciousness of almost all claims made in the name of Science! are not even this subtle. But I think it important to get a grip on what scientific claims, questions and evidence ought to look like.
- Of course, he was ‘wrong‘:
In a commencement address given at the California Institute of Technology (Caltech) in 1974 (and reprinted in Surely You’re Joking, Mr. Feynman! in 1985 as well as in The Pleasure of Finding Things Out in 1999), physicist Richard Feynman noted:
We have learned a lot from experience about how to handle some of the ways we fool ourselves. One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It’s a little bit off because he had the incorrect value for the viscosity of air. It’s interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan’s, and the next one’s a little bit bigger than that, and the next one’s a little bit bigger than that, until finally they settle down to a number which is higher.
Why didn’t they discover the new number was higher right away? It’s a thing that scientists are ashamed of—this history—because it’s apparent that people did things like this: When they got a number that was too high above Millikan’s, they thought something must be wrong—and they would look for and find a reason why something might be wrong. When they got a number close to Millikan’s value they didn’t look so hard. And so they eliminated the numbers that were too far off, and did other things like that …