Making the Sausage: Building and Using Models in the Real World (pt 2)

Something must be done!

THIS is something!

This must be done!

This little unfunny joke reminds me of one of the greatest of Sun Tzu’s teachings: the greatest victory is the battle not fought. When you can achieve your ends without having to roll out the army, that’s the real victory. But, the great Chinese general also points out, there’s no glory in that, since the less enlightened won’t be able to see the victory in the battle not fought. It takes the wisdom and confidence of a great general to have the guts to not fight the battle.

Another bit of real-world experience: the less people know about modelling, the more they expect out of it. The completely ignorant want magic, and, not understanding how models work, get put out when you can’t deliver. In the business world, there’s a great push for predictive analytics. Wouldn’t it be nice to be able to know which deals would go into default before you did them? Or how money costs were going to change over the next decade? Or even what the federal tax rates and rules were going to be?

Like any other model, predictive analytics are an expression of the prejudices of the model builder. Only when what happens in the real world show that the assumptions built into the model conform enough to reality to be useful do those assumptions become anything more than prejudices. Note I’m here talking about *predictive* analytics. The point is that only when they are no longer truly predictive are they be useful -only when they fundamentally say: assuming what has happened in the past continues to happen or happens again, THEN this is what will happen – are they any more than prejudices.

Predicting what will happen based on unknowns, based on things that have never happened before, is fantasy. The Drake Equation (1) is fantasy. Anthropomorphic Global Warming is essentially a fantasy (2). The giant unknowns – what factors causes climate in general – dwarf any supposed science behind a relatively minor greenhouse gas. It could could gain some reality if clear data existed over meaningful periods of time showing the correlation and – big part – accounting for all the other factors, known and unknown, that have caused the climate to change incessantly over a couple billion years. Since that hasn’t happened, and is unlikely to happen any time soon, what we’re left with are prejudices dressed up in math.

But something must be done! We’re all gonna die!

In the business world, the check upon crystal-ball promises is money and reality. The most success has been had in taking well-understood marketing data – through years of gumshoe-level marketing research, we know our ideal prospect has these buying habits and characteristics – then applying Big Data to finding more of those customers. The more you depart from such processes, the riskier it becomes. It certainly can work, but I’ll bet the purveyors of Big Data make sure we hear a lot about success stories, and not so much about the failures. At any rate, the money will soon dry up if there are not results. (3)

But in the world at large, especially if a government is bankrolling the ‘research’ behind a model, these restrictions don’t apply. Ferguson, while still trailing by miles, is now in the race with Erlich for the ant-Cassandra crown: the more wrong he is, the more credible he is assumed to be.

Governments, as giant bureaucracies run, according to Pournelle’s Iron Law, by careerist bureaucrats, will never willingly let a crisis go to waste. Something must be done, and they’ll find the something that most involves government action.

We had no Sun Tzu. A great victory could have been won by doing next to nothing, but an even greater defeat was delivered in its stead. There is no honor to be had, despite the claims of back-patting fear mongers.

Another relevant Sun Tzu saying: If you know yourself and know your enemy, victory is assured; if you know yourself and don’t know the enemy, you may win the battle half the time; victory is impossible if you know neither yourself nor the enemy. Since the enemy – COVID 19 – was (studiously) not known before the war was declared, and who ‘we’ are was unknown, victory as any sane person of good will would understand it was impossible from the get go.

  1. TL;DR version:

“This serious-looking equation gave SETI a serious footing as a legitimate intellectual inquiry. The problem, of course, is that none of the terms can be known, and most cannot even be estimated. The only way to work the equation is to fill in with guesses. And guesses-just so we’re clear-are merely expressions of prejudice.

“Nor can there be “informed guesses.” If you need to state how many planets with life choose to communicate, there is simply no way to make an informed guess. It’s simply prejudice.”

2. From the same source:

“Just as the earliest studies of nuclear winter stated that the uncertainties were so great that probabilities could never be known, so, too the first pronouncements on global warming argued strong limits on what could be determined with certainty about climate change.

“The 1995 IPCC draft report said, “Any claims of positive detection of significant climate change are likely to remain controversial until uncertainties in the total natural variability of the climate system are reduced.” It also said, “No study to date has positively attributed all or part of observed climate changes to anthropogenic causes.”

“Those statements were removed, and in their place appeared: “The balance of evidence suggests a discernible human influence on climate.” What is clear, however, is that on this issue, science and policy have become inextricably mixed to the point where it will be difficult, if not impossible, to separate them out. It is possible for an outside observer to ask serious questions about the conduct of investigations into global warming, such as whether we are taking appropriate steps to improve the quality of our observational data records, whether we are systematically obtaining the information that will clarify existing uncertainties, whether we have any organized disinterested mechanism to direct research in this contentious area.

“The answer to all these questions is no. We don’t.”

3. There are crazy people in business, of course, as in any other group of people, so sometimes the money for a crazy project doesn’t run out before the money in general runs out. But not usually.

Author: Joseph Moore

Enough with the smarty-pants Dante quote. Just some opinionated blogger dude.

One thought on “Making the Sausage: Building and Using Models in the Real World (pt 2)”

  1. World climate models have some other interesting issues.

    Why do experts think that, say, the fluids assumptions are correct? Presuming that they do.

    Well, we have these fundamental equations of fluid dynamics, the science is settled, right?

    First, on the theory side, look at the history of the field, and you notice that there were major theoretical breakthroughs just last century. How likely is it that no more remain? Ask a fluids guy what the deal is with turbulent flow. Some will give you an explanation, sure. Others may tell you that they are not entirely satisfied with the theory.

    Second, on the practical side, are those processes used for other fluids problems with lives at stake? For example, do researchers in computational fluid mechanics for engineering applications count the work done when it matches theory, or when it better matches experimental data of that exact flow situation? Are solutions considered generally applicable, or are they very specific? Can we infer that there are flow phenomena in the atmosphere that would be difficult to experimentally replicate?

    Well, if there were no gravity, the atmosphere would not hang around. Gravity is important. At the scale of the whole planet, gravity is radially inward. How would you simulate that at a laboratory scale?

    Now, a real expert may have an answer to that. Maybe they can even point to data from a good set of experiments.

    If you have a little exposure to one of the subfields used in a model, you may have cause to ask, wait, what about this bit? Starting the conversation with “shut up, we have the answers now” is a blinking red warning light. The people who can give you useful answers about a science question are the people who can and will tell you something about their own limits. If ‘just do what the book says’ is good enough, just about anyone can read a book. Okay, understanding and applying a given book is more difficult. But it is much harder to either do ‘comparing these books, we see such agreements and disagreements’ or ‘from my experience, the book probably has this bit wrong’ well.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: