POLS 3220: How to Predict the Future
Suppose the class is trying to predict whether the Federal Reserve will cut interest rates at its next meeting.
Three students submit briefings, and all three say 75%.
What should you predict after observing those three forecasts?
Previously, we considered taking the median forecast (i.e. majority rule).
How best to aggregate individual forecasts into a group forecast?
I’ll try to convince you that, if three individual forecasts are at 75%, the best aggregation is probably >75%.
We’ll motivate this idea from the perspective of Bayesian updating.
Suppose you’re writing a briefing about the interest rates question.
At first, you have no idea what will happen (prior odds = 1:1).
Then you find a statement from a Fed official expressing concern about unemployment.
Historically, he makes statements like this three times more often before an interest rate cut than he does otherwise.
Problem: What should your new forecast be, after observing this evidence?
Remember Bayes Rule:
\[ \text{Posterior Odds} = \text{Prior Odds} \times \text{Strength of Evidence} \]
Remember Bayes Rule:
\[ \underbrace{\text{Posterior Odds}}_{\text{New Forecast}} = \underbrace{\text{Prior Odds}}_{\text{Old Forecast}}\times \underbrace{\text{Strength of Evidence}}_{\text{Likelihood Ratio}} \]
Now suppose you read that JP Morgan’s analysts are predicting an interest rate cut.
Historically, this appears to have the same strength of evidence as the Fed official’s statements.
Bayes Rule handles this problem exactly like the first one!
This process of Bayesian updating can proceed indefinitely.
For each piece of evidence:
Consider the three students writing briefings. All three start with prior odds 1:1.
Student A only finds the statement by the Fed official (3:1 likelihood ratio).
Student B finds the JP Morgan report (3:1 likelihood ratio).
Student C finds a third piece of evidence (also 3:1).
If nobody shares information with each other, what would each student predict?
If everyone shared their information, what would they predict (using Bayesian updating)?
This is the intuition behind extremizing predictions.
If a group of forecasters have independent pieces of information, they should become more confident if they shared what they know.
We can simulate this process by making everyone’s forecast a bit more confident before taking the average.
How much you extremize depends on how much independent information you think the crowd has.
If every individual has fully independent evidence informing their prediction:
\[ o_{group} = o_1 \times o_2 \times o_3 \times \ldots \times o_n \]
This is full Bayesian updating; multiply everyone’s odds together to get the posterior.
But that independence assumption is wildly optimistic!
Forecasters are usually relying on the same sources of information (Google searches, chatbots, newspapers, etc.)
More reasonable assumption: the crowd has some independent pieces of information.
If each individual has some independent information:
\[ o_{group} = (o_1 \times o_2 \times o_3 \times \ldots \times o_n)^\frac{\alpha}{n} \]
\(\alpha\) represents the amount of independent information.
\(\alpha = n\): Everyone has completely independent information. Group forecast is full Bayesian updating.
\(\alpha = 1\): No one has independent information. Group forecast is just the average individual forecast.
Truth is probably somewhere in the middle. For superforecasters, Satopää et al. (2014) use \(\alpha = 2.5\).
Five forecasters predict 40%, 50%, 60%, 75%, and 75%.
The median forecast is 60%.
The extremized forecast \((\alpha = 2.5)\) is:
\[ (2:3 \times 1:1 \times 3:2 \times 3:1 \times 3:1)^\frac{2.5}{5} = (9:1)^\frac{1}{2} = 3:1 = 75\% \]
This isn’t the sort of math I’d ask you to perform on an exam.
I just want you to have it in your toolbox.
Confident predictions are justified if multiple, independent pieces of evidence point towards the same conclusion (Bayesian updating, dragonfly-eyed foxes).
When reading briefings, consider whether each author is bringing a different perspective, or if everyone is relying on the same evidence.
Organize teams so that they share information rather than just sharing conclusions.