Communicating Uncertainty in Policy Analysis

In a series of papers Charles F. Manski of Northwestern University discuss the issue of presenting uncertainty to policy makers. See "Communicating uncertainty in policy analysis" (2019) PNAS. This argues that policy analysis combines assumptions and data to draw conclusions. Holding fixed the available data, stronger assumptions may yield stronger conclusions. At the extreme sufficiently strong assumptions with yield certain conclusions. A fundamental difficulty is to decide what assumptions to maintain. Stronger assumptions yield conclusions that are more powerful but less credible.

Policy analysis can illuminate the logic by posing alternative assumptions and determining the conclusions that follow. However, people have a tendency to want to sacrifice credibility to obtain strong conclusions. Many policy makers and members of the public resist facing up to uncertainty, perhaps if we explain it better they will be more willing to face up to it.

We tend to think of logic going going from assumptions and data to conclusions. However, it can be informative to start from a conclusion and ask: what would we need to assume, given the data we have, to believe this conclusion? Sometimes economic theory can help us choose between the various assumptions required to yield different conclusions. This can be scientifically valid, but can also be confused with advocacy if it is not done objectively and communicated well.

But sometimes economic theory does not help us choose between alternative assumptions, and we need to communicate clearly when that is the case. False certainty will lead to bad policy formation. If policy makers understand the nature of the uncertainty they can:

• focus research on resolving key areas of uncertainty
• make policy that is adaptive or flexible, to deal with possible different eventualities or outcomes

You want to convey whether the facts and figures you present are: 

• robust, from an authoritative source, based on extensive research and well measured
• speculative, controversial, very approximate

Are they measure precisely, or is there a large degree of statistical uncertainty? 

Do they depend on some key assumption, or will they vary in different states of the world?