Be wary of computer models and subsequent interventions

Be wary of computer models and subsequent interventions

Modeling can analyze data about complex systems and make predictions. But the predictive value of computer modeling is hampered by the fact that models are only as good as the assumptions and statistics behind them.

Models have become ubiquitous in government planning and policy-setting. But that carries built-in dangers connected with the inherent problems with models and the predictions based on them, particularly if the policies recommended are far-sweeping and damaging in themselves.

The more catastrophic the prediction, the more likely a government or a populace is to become afraid, and to justify large-scale interventions that can negatively impact liberty, the economy, and quality of life in general. And all for what? To avert a catastrophe that never would have happened in the first place? A lot of people – even those who are generally science-oriented – have come to distrust the models and consider them deeply flawed or even deceptive, an excuse for government to clamp its hand ever more tightly around us.

If a government doesn’t follow the modelers’ suggestions about AGW, for example, and yet the catastrophe doesn’t appear when predicted, the forecasts are often just pushed into the future. If a government does follow the modeler’s suggestions and nothing catastrophic happens, those who advocated the draconian interventions can claim, “See, if you hadn’t listened to us it would have been absolutely terrible, just as we predicted. Now you must always listen to us, because we are correct.”

And yet there’s only one earth. We cannot do a controlled experiment because there is no second planet to act as a control. And although we can try our best to crunch the numbers, how can we know for certain whether the intervention made things better or not, or was necessary at all?

Pandemics are similar in that modeling and predictions are involved. But with pandemics we have more opportunity for checking our work. We can compare different countries and different interventions during the same pandemic, although that can take not just years but a century or more; for example, they’re still trying to decide whether certain cities in the US instituted interventions in 1918 that made an ultimate difference in their rate of death from flu. After all, it’s not as though all cities in a certain country, or all countries on earth, are the same in terms of a host of factors that might make them more or less susceptible to a particular illness. Researchers have ways of adjusting for those differences. But the methods to do so are far from perfect. And yet we really need to know.

But that’s mostly ex post facto, and enormously consequential decisions must be made in real time. Choosing a model and deciding whether to intervene and how to intervene is very hard. Following advice that features a worst case scenario, just in case, can lead to enormous interventions that hurt a city or country (or world) and may have been unnecessary.

In the olden days, science didn’t have computer models, and people were at the mercy of epidemics or pandemics that ravaged their countries and sometimes much of the world. But even then, in their desperation they tried to figure out what to do:

Hippocrates and Galen are colossal figures in the history of medicine [offered guidance for plague], rendered in Latin as ‘Cito, Longe, Tarde,’ which translates as ‘Leave quickly, go far away and come back slowly.’…

When the Black Death spread through Italy in late 1347, some ports began turning away ships suspected of coming from infected areas. During March the following year, authorities in Venice became the first to formalise such protective actions against plague, closing the city’s waters to suspect vessels, and subjecting travellers and legitimate ships to 30 days’ isolation. This period was extended to 40 days some years later – hence the term quarantine. Further regulations established remote cemeteries for plague victims who in turn were collected, transported and buried in accordance with defined rules. But these measures were too little, too late. Plague took hold and Venetians died in their tens of thousands…

Other Italian cities tried similar measures. Further inland, in May 1348 the northern city of Pistoia introduced wide-ranging laws affecting many aspects of daily life. Restrictions on imports and exports, travel, market trading and funerals were all brought in, but again to no effect. At least 70% of the population died. But by contrast, another northern city, Milan, avoided a major outbreak. Whether this was due to control measures taken by city authorities, including sealing up three houses (with the occupants inside) after plague was discovered there, is debatable. The Milanese authorities could certainly be firm. From 1350 they decreed that all future plague victims and those nursing them would be isolated in a designated pesthouse built outside the city walls.

I’m grateful for modern science. But unfortunately, we may not be as good as we think we are – and certainly not as good as we would like to be – at predicting the effects of the major interventions we employ in our attempts to control the forces of nature.

[Neo is a writer with degrees in law and family therapy, who blogs at the new neo.]

Donations tax deductible
to the full extent allowed by law.