Evidence and the Right Questions

When we last left off, we were discussing claims and evidence. Now let’s talk about the quality of claims, evidence, and the relationship of claims and evidence. This it probably Part 1 – topic spiral potential: high.

To cut to the chase: a reasonable, useful claim is specific, expressed in unambiguous terms, and subject to logical and real-world contradiction.

Thus, reasonable, useful evidence addresses specific claims, according to the rules of logic.

This all may seem pedantic nonsense. If so, you will find real science is all about pedantic nonsense. Ask a scientist a simple question: what is the boiling point of water? and, if he is answering as a scientist, you will get:

  • discussions about what a state change is, energy thresholds, margins of error, and limits of observation;
  • A laundry list of conditions that affect the observed boiling point of water: air pressure, purity of the water;
  • THEN he might say: 100C, GIVEN all the definitions, conditions, and caveats listed above.

The boiling point of water is about as simple a scientific question as one can ask. See Millikan’s classic oil drop experiment (my favorite, and the fanciest experiment I’ve every personally done) for something a little more complicated. To calculate the charge of an electron, Millikan and Fletcher had to define, develop and measure a whole bunch of things, e.g., the size and mass of aerosol oil droplets and the viscosity of air (which changes with temperature and pressure). They needed to design and build a device that 1) created tiny oil droplets; 2) generated electrons in such a way that some of them would stick to those oil droplets; 3) provided a consistent, measurable way to observe the oil droplets thus created; 4) had a magnetic field of known strength that they could turn on and off at will. THEN you spend hundreds of hours (in addition to the hundreds you spent coming up with the experimental concepts and building and perfecting the device) risking blindness to gather thousands of observations.

Millikan did all that, and a ton of math, then got to say that the charge of the electron is 1.5924(17)×10−19 C (1) and collect his Nobel Prize.

In the real world, few people understand the question: what is the charge of an electron? let alone feel any need to know the answer.

A scientific claim is a claim that answers a scientific question. (I’m a regular Obvious Oscar today!) If the question itself does not go through the refining and defining required to hammer it into scientific shape, cleaning up as much as possible all ambiguities and and establishing the limits and conditions, then the pseudo-scientific claim that science has answered such a question is, and must be, wrong.

The above is a round-about way of addressing the nature of science as discussed here for a decade or more. Science, as John C. Wright points out often, is the study of the metrical properties of physical objects. If the question does not concern the measurement of the properties of something you can see, hold in your hand, smell, taste, hear – then it’s not a scientific question. Note: this does not mean your question is unimportant or wrong, merely that you’re not going to be able to use science to answer it. Most of life’s really important questions – should I ask her to marry me? what is the right thing to do? how should I spend my life? and so on – are not science questions. We have to come up with other ways to answer them.

It should be clear at this point that scientific evidence must be weighed by how well, if at all, it addresses a well formed scientific question. Badly formed or categorically wrong questions cannot, as in, CANNOT be answered scientifically. Science will not tell me if I should ask this particular woman to marry me; science has nothing to say about the proper course of action for any human decisions. Getting an ought from an is is difficult, if, indeed, the impossible can be called difficult. (That’s an Aristotle joke, there.)

Thus, people are making a categorical error when they claim to be ‘following the science’ when the do or promote actions. There’s always more to the question. Ex: if someone is injected with these chemicals, they will die. Therefore, IF we don’t want a particular someone to die, we should not inject them with these chemicals. So, do we want them to die? Are they an innocent child, or a serial killer of innocent children on death row? Could have different answers. IF the person OUGHT to die is not a question science can answer.

To be convincing or even relevant scientifically, evidence requires a good, clean scientific question to be run up against. Take an example I’ve used before: I saw a report once that a certain migratory butterfly population had decreased 87.3% (say. Numbers are for illustration only.) The obvious scientific question this ‘evidence’ would be addressing is: are there fewer butterflies of this particular type in a specified time period as opposed to another specified time period? Simply putting the question in that format should suggest the conditions and definitions needed in order to evaluate evidence, if any:

  1. How is the counting of butterflies being done?
  2. Where?
  3. When?
  4. How is the accuracy of the counting assured? (This means, in Feynman’s classic formulation, that the makers of the claim/presenters of the evidence are required by the honesty implicit in the pursuit of science to list any possible ways they can think of that their conclusions, methods, or data could be wrong.)

In turn, these questions intended to clear up and make scientific the more general question do, themselves, raise questions. And here’s the point of this exercise: if the study or report authors or claimants cannot show that they have done the thought-smithing needed to define and clarify the question they claim to be addressing, then, put bluntly, it’s not science. In the above example, at least, the ‘researchers’ would need to assure us that:

  1. Where they are looking for the butterflies is where the butterflies are – namely, that they didn’t simply take another route, or take the usual route at a different time. In other words, that their count is in fact a count that includes all the relevant butterflies.
  2. How they counted those thousands of butterflies is at all accurate.

And so on.

In the real world, the speciousness of almost all claims made in the name of Science! are not even this subtle. But I think it important to get a grip on what scientific claims, questions and evidence ought to look like.

  1. Of course, he was ‘wrong‘:

In a commencement address given at the California Institute of Technology (Caltech) in 1974 (and reprinted in Surely You’re Joking, Mr. Feynman! in 1985 as well as in The Pleasure of Finding Things Out in 1999), physicist Richard Feynman noted:

We have learned a lot from experience about how to handle some of the ways we fool ourselves. One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It’s a little bit off because he had the incorrect value for the viscosity of air. It’s interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan’s, and the next one’s a little bit bigger than that, and the next one’s a little bit bigger than that, until finally they settle down to a number which is higher.
Why didn’t they discover the new number was higher right away? It’s a thing that scientists are ashamed of—this history—because it’s apparent that people did things like this: When they got a number that was too high above Millikan’s, they thought something must be wrong—and they would look for and find a reason why something might be wrong. When they got a number close to Millikan’s value they didn’t look so hard. And so they eliminated the numbers that were too far off, and did other things like that …

Author: Joseph Moore

Enough with the smarty-pants Dante quote. Just some opinionated blogger dude.

5 thoughts on “Evidence and the Right Questions”

  1. “…a reasonable, useful claim is… expressed in unambiguous terms…”

    Which means, of course, that, so long as the biological term “species” is universally acknowledged to be highly ambiguous, no claim to have discovered the mechanism of speciation can be reasonable and useful. Funny how guys like Feynman never seem to talk about that.

    “Ask a scientist a simple question… and you will get, etc., etc., etc.”

    This makes me wish I could embed images in these messages. But, then again, you’ve probably already seen that classic “Foxtrot” strip in which a simple word problem about train speeds receives the Jason treatment. (“…assuming that Stations A and B are in the same time zone; assuming the times occur on the same day; assuming that 180 miles is the distance along the curved surface of the Earth and not ‘as the mole digs’, so to speak…”)

    1. Sure, But there’s also the ‘twilight does not disprove night and day’. You’re in for a load of trouble distinguishing one subspecies of cichlid from another, but dolphins, tuna, and starfish are just different. If we posit common descent from some distant ancestor, then speciation took place someplace along the line, however difficult is was to see at the time.

      To take a more pertinent example: horses and donkeys sure look different, yet, genetically, they differ only slightly. But that difference is enough that offspring of a horse and donkey is almost always sterile. So, I doubt anyone is too worried over the existence of differences which are slight on one level – warm-blooded herbivorous quadrupeds with hooves, etc., yet decisive on another – there aren’t any self-sustaining populations of horse/donkey hybrids. So we call them different species with a fair amount of confidence.

      1. I take this reply as meaning that, whether or not the biological term “species” – or, for that matter, “taxon” generally – has ever been wholly unambiguous in the mouth of any human being, still it is intuitively certain that it corresponds to some objective reality. And I don’t dispute this, but it doesn’t really affect my basic point – except, perhaps, to make it even starker, since it implies that one could (in theory) determine and define the reality in question so as to *make* the term wholly unambiguous. Which is to say that Hugo de Vries’s claim, while not reasonable and useful yet, could conceivably be made so with some tinkering – and yet guys like Feynman, who would ordinarily be the first in such a case to insist that the claimant’s supporters go back and do the tinkering before asking anyone to listen to them, run up against this particular claim and fall conspicuously silent. Is interesting, no?

        (I might also note that, in a sense, twilight *does* disprove night and day. That is, it proves – or indicates, rather – that there is no *qualitative* difference between the two states, both being matters of one’s position on the same rotational path. Which makes it rather an awkward metaphor to use in the present context, unless you’re actually prepared to assert that there is likewise no qualitative difference between hippos and cornstalks.)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s