Kathryn Cochran Interview

In this episode (recorded 1/23/18), we interview Good Judgment Chief Scientist Kathryn McNabb Cochran about forecasting question design. Dr. Cochran leads Good Judgment’s question development team and helped design Good Judgment’s new Future of Europe Index.

We talked with Dr. Cochran how forecasting can make organizations more agile; about how to design clear, meaningful forecasting questions; about how forecasting makes us smarter consumers of news; about the value of forecasting question clusters; about how we could improve the forecasting ability of US intelligence; and about the Nicaragua Canal and little green men.

You can read Dr. Cochran and Cdr. Tozzi’s Center for a New American Security essay, “Getting It Righter, Faster,” here. The announcement for Good Judgment’s Future of Europe Index is here.

You can find NonProphets on BlubrryiTunesStitcher and Google Play. As always, if you have questions, ideas for future episodes, or would like to suggest a possible black swan event, you can use our contact page or e-mail us at (nonprophetspod [at] gmail.com). If you enjoy this podcast, please rate us and recommend us to your friends.

Sichere Prognosen (Bruno Jahn Interview)

In this episode (recorded 11/1/17), we interview superforecaster Bruno Jahn. Bruno is a forecasting consultant with an M.A. in Islamic/Middle Eastern Studies from the Freie Universität Berlin.

We begin by talking about how Bruno got involved with Good Judgment and how he became a superforecaster. Then we go through a number of Good Judgment Project forecasting questions we did poorly on and discuss whether our forecasts were really inaccurate or whether we were just unlucky. We go on to consider whether expertise can sometimes be an obstacle to forecasting.

Next Bruno tells us about his forthcoming book on forecasting, Sichere Prognosen in unsicheren Zeiten (“Certain Forecasts in Uncertain Times”), which comes out in April 2018. Bruno says that modern forecasters play a similar social to Roman augurs and talks about the value of “common-sense Bayesianism”. He also argues that serious forecasting can serve as an antidote to fake news. We close the podcast by talking about what supeforecasters are like. whether we will see a “Jamaica coalition” in Germany, and why center-left parties seem to be collapsing in Europe.

You can find NonProphets on BlubrryiTunesStitcher and Google Play. Links to our other superforecaster interviews are here. As always, if you have questions, ideas for future episodes, or would like to suggest a possible black swan event, you can use our contact page or e-mail us at (nonprophetspod [at] gmail.com). If you enjoy this podcast, please rate us and recommend us to your friends.

Welton Chang Interview

In this episode (recorded 9/20/17), we interview superforecaster Welton Chang. Welton is a Ph.D. candidate in psychology at Penn in Philip Tetlock and Barb Mellers’ Good Judgment Laboratory. He served in Iraq and South Korea as an intelligence officer in the US Army and as analyst for the Department of Defense.

We start by talking with Welton about how he got involved the Good Judgment Project and the Good Judgment Laboratory. Then we talk about uncertainty in forecasting and Vizzini’s Princess Bride conundrum, the value of algorithmic forecasts (which we also talked about on another recent podcast), the limits of modern warfare, and whether Kim Jong-un and Donald Trump are rational actors.

We go on to talk about designing training materials for Good Judgment Project forecasters, IARPA’s CREATE program on improving analytic reasoning, avoiding groupthink, the importance of diversity in forecasting, and the art and practice of applying Bayes’ Theorem. We close with a shout out to Welton’s rescue cats, Percy and Portia.

You can read Welton Chang’s essay on the Iran deal, “Go Set a Watchdog on Iran” here. You can read Evan Osnos’ “The Risk of Nuclear War With North Korea” here. You can follow Welton (@WeltonChang) on Twitter here. You can follow Welton’s cats (@percyandportia) on Instagram here.

You can find NonProphets on BlubrryiTunesStitcher and Google Play. Links to our other superforecaster interviews are here. As always, if you have questions, ideas for future episodes, or would like to suggest a possible black swan event, you can use our contact page or e-mail us at (nonprophetspod [at] gmail.com). If you enjoy this podcast, please rate us and recommend us to your friends.

Can We Trust AI Judgments?

We start this episode (recorded 8/16/17) by announcing the Hybrid Forecasting Competition (HFC). Good Judgment is collaborating with IARPA to run a tournament designed to study whether human forecasters can work with machine systems to improve geopolitical forecasting. You can sign up to volunteer as a forecaster in the competition here.

We then interrupt the podcast briefly to explain that some of the podcast audio was lost due to what we believe was an attack by a rogue artificial intelligence named “Super Bert”.

Next we discuss to what extent we can believe the forecasts of AI forecasters. Robert argues that it can be hard even to know when trust human forecasters and recommends a recent Harvard Business Review interview with former US National Coordinator for Security, Infrastructure Protection and Counter-terrorism Richard Clarke on “Cassandras”, whose warnings of disasters aren’t believed. Atief makes the case that our unfamiliarity with the behavior of AI should it harder to trust AI when it gives surprising results.

We go on to talk about the “value-alignment problem” in AI design, Microsoft’s racist chatbot, Isaac Asimov’s three laws of robotics. We then return to the question of how much we can trust AI judgments but our discussion ends suddenly when Super Bert destroys the rest of our recording.

You can find NonProphets on BlubrryiTunesStitcher and Google Play. As always, if you have questions, ideas for future episodes, or would like to suggest a possible black swan event, you can use our contact page or e-mail us at (nonprophetspod [at] gmail.com). If you enjoy this podcast, please rate us and recommend us to your friends.

Elaine Rich Interview

In this episode (recorded 7/12/17), we interview superforecaster and pharmacist Elaine Rich. Elaine started with Good Judgment in the first year of IARPA-ACE forecasting tournament after reading about the project on FiveThirtyEight. She was interviewed by NPR in 2014 about her experience with Good Judgment in a segment called “So You Think You’re Smarter Than a CIA Agent”.

We start by talking with Elaine about her experience forecasting at the beginning of the Good Judgment Project, the advantages of forecasting anonymously, why she doesn’t like forecasting subjects close to her professional expertise, and whether accurate forecasting really matters. We also talk about whether forecasting ability is something innate you are born with or something you learn or can train, whether gut instincts are any good, and the value of good journalism for forecasting. Elaine ends the interview by recommending the Joni Mitchell album Hejira—and in particular the song “Amelia”—as forecasting music.

You can find NonProphets on BlubrryiTunes, Stitcher and Google Play. Links to our other superforecaster interviews are here. As always, if you have questions, ideas for future episodes, or would like to suggest a possible black swan event, you can use our contact page or e-mail us at (nonprophetspod [at] gmail.com). If you enjoy this podcast, please rate us and recommend us to your friends.

Roman Hagelstein Interview

UPDATED: German speakers can read Roman Hagelstein’s profile in Welt am Sonntag.

In part one of our interview with German economist and superforecaster Roman Hagelstein (recorded 4/14/17), we discuss how Roman’s German background affects his forecasting, how he gamed one of the Good Judgment Project’s forecasting markets, the value of “extremizing” forecasts, and why civil wars tend to last a long time.

In part two of our interview with Roman, we discuss American policy in Syria and what Germans think about German Chancellor Angela Merkel. Then we talk about the Brier Fund, a private investment fund run by superforecasters, and about the challenge of forecasting financial markets. After a digression on who will win Most Valuable Player in the NBA this year, we finish up by talking about the chance of a crisis related to North Korea in the near future.

You can find NonProphets on BlubrryiTunes and Google Play. Links to our other superforecaster interviews are here. As always, if you have questions, ideas for future episodes, or would like to suggest a possible black swan event, you can use our contact page or e-mail us at (nonprophetspod [at] gmail.com. If you enjoy this podcast, please rate us and recommend us to your friends.

Episode 23: I Have No Information

We start this episode (recorded 3/22/17) by talking about what Scott learned teaching forecasting at a Good Judgment training workshop in Sweden. We discuss the value of the Delphi method in combining the individual judgment of forecasters while also allowing them to draw on the wisdom of the crowd. We also talk about the importance of outliers, how to know if we are undergoing a paradigm shift, and the danger of giving too much weight to private information.

Next we talk about Robert’s long weekend on the island of Hawai‘i seeing lava flow into the ocean, stargazing on the top of Mauna Kea, and swimming with honu. Then we consider the prospects for a Republican healthcare bill. We talk about how we try to be objective in a polarized political environment and about whether the Trump administration is credible. We close the podcast by hearing about Atief’s trips to New York to go to museums and see the Groundhog Day musical and to Oklahoma City to see the Thunder play the Warriors.

You can find NonProphets on BlubrryiTunes and Google Play. As always, if you have questions, ideas for future episodes, or would like to suggest a possible black swan event, you can use our contact page or e-mail us at (nonprophetspod [at] gmail.com. If you enjoy this podcast, please rate us and recommend us to your friends.

Episode 17: Michael Story Interview

In episode 17 (recorded 1/26/17), we interview Good Judgment director and friend of the podcast Michael Story. Michael is a British superforecaster with an MSc. in public policy research from the London School of Economics.

We talk with Michael about his approach to forecasting and about his experience training forecasters. We go on to discuss the class politics of Brexit, the rise of nationalism in Europe, what happens if the US support for NATO weakens, and who would win in a rematch between the UK and Argentina. We close by considering whether there is realistic chance that a country uses a nuclear weapon in 2017.

The article on training forecasters Michael mentioned is

Mellers, Barbara, et al. “Psychological strategies for winning a geopolitical forecasting tournament.” Psychological Science 25.5 (2014): 1106-1115.

You can also find NonProphets on iTunes and Google Play. Links to our other superforecaster interviews are here. As always, if you have questions, ideas for future episodes, or would like to suggest a possible black swan event, you can use the contact page or e-mail us at (nonprophetspod [at] gmail.com). And if you enjoy this podcast, please rate us on iTunes and recommend us to your friends.