Episode

Why Great Data Still Leads to Bad Decisions (And How to Fix It) with Amy Edmondson and Michael Luca

Amy Edmondson (Harvard Business School) and Mike Luca (Johns Hopkins) join High Signal to unpack what actually drives good decisions in data‑rich organizations. Using contrasts like the Bay of Pigs vs. the Cuban Missile Crisis and product cases such as Airbnb’s work on measuring discrimination, they show how decision quality tracks conversation quality—framing options, surfacing uncertainty, and challenging assumptions. We cover common failure modes (correlation vs. causation, anchoring, hierarchy, false precision), practical meeting designs that raise the signal, and where algorithms and LLMs help or hinder human judgment.

August 5, 2025

Listen on

Guest

Amy Edmondson

Harvard Business School

Amy C. Edmondson is the Novartis Professor of Leadership and Management at the Harvard Business School, a chair established to support the study of human interactions that lead to the creation of successful enterprises that contribute to the betterment of society.

Edmondson has been recognized by the biannual Thinkers50 global ranking of management thinkers since 2011, and most recently was ranked #1 in 2021 and 2023; she also received that organization’s Breakthrough Idea Award in 2019, and Talent Award in 2017.  She studies teaming, psychological safety, and organizational learning, and her articles have been published in numerous academic and management outlets, including Administrative Science Quarterly, Academy of Management Journal, Harvard Business Review and California Management Review. Her 2019 book, The Fearless Organization: Creating Psychological Safety in the Workplace for Learning, Innovation and Growth (Wiley), has been translated into 15 languages. Edmondson’s latest book, Right Kind of Wrong (Atria), builds on her prior work on psychological safety and teaming to provide a framework for thinking about, discussing, and practicing the science of failing well. First published in the US and in the UK (Penguin) in September, 2023, the book is due to be translated into 24 additional languages, and was selected for the Financial Times and Schroders Best Business Book of the Year award.

‍

Guest

Michael Luca

Johns Hopkins University

Michael Luca is a professor and the director of the Technology and Society Initiative at the Johns Hopkins University, Carey Business School, and a research associate at the NBER. Professor Luca's research, teaching, and advisory work focuses on the design of online platforms, and on the ways in which data can inform managerial and policy decisions.His research has been published in academic journals including the Journal of Economic Perspectives, Management Science, the Proceedings of the National Academy of Sciences, American Economic Review: Papers and Proceeding, the American Economic Journal: Applied Economics, and the American Economic Journal: Microeconomics.He has also written about behavioral economics and online platforms for media outlets including The Wall Street Journal, Harvard Business Review, The Atlantic, Wired, and Slate. His research has been written about in a variety of media outlets including The Wall Street Journal, New York Times, New Yorker, Atlantic, Economist, Washington Post, Financial Times, Guardian, Huffington Post, Harvard Business Review, Time, USA Today, Boston Globe, LA Times, San Francisco Chronicle, Fortune, Mashable, GQ, Wired, and Vox.

HOST

Hugo Bowne-Anderson

Delphina

Hugo Bowne-Anderson is an independent data and AI consultant with extensive experience in the tech industry. He is the host of the industry podcast Vanishing Gradients, a podcast exploring developments in data science and AI. Previously, Hugo served as Head of Developer Relations at Outerbounds and held roles at Coiled and DataCamp, where his work in data science education reached over 3 million learners. He has taught at Yale University, Cold Spring Harbor Laboratory, and conferences like SciPy and PyCon, and is a passionate advocate for democratizing data skills and open-source tools.

Key Quotes

Key Takeaways

Decision quality is conversation quality.
Bay of Pigs vs. Cuban Missile Crisis shows the delta wasn’t data: iit was the rigor of debate, exploration of options, and pressure-testing assumptions.

‍

Design the decision, not just the analysis.
Good outcomes come from structured meetings: name the uncertainty, surface dissent, switch sides, and expand the option set before choosing.

‍

Measurement choices shape strategy.
What you track becomes what you optimize. Airbnb’s shift (measuring discrimination, adding guardrails, and changing product flows) shows how metrics redirect behavior.

‍

Causality is earned, not assumed.
Correlation doesn’t settle managerial bets. Leaders must ask how the claim was identified (experiment? quasi-experiment?), what confounders remain, and what would change the decision.

‍

Treat uncertainty as a first-class input.
Ranges, confidence intervals, and priors beat single point estimates. Decide for robustness across plausible outcomes, not for a fragile best guess.

‍

Culture can overpower evidence.
Anchoring, hierarchy (the HiPPO), and confirmation bias routinely swamp data. Create norms and language to call these out in the room before they harden into decisions.

‍

Weak signals deserve investigation.
“Not significant” isn’t “not real.” Probe subgroup effects, moderators, and time horizons: early warnings often hide in heterogeneity and short windows.

‍

Generalize with care.
Ask what worked, why it worked, and where the mechanism holds. Internal validity is not external validity; portability requires understanding the “why,” not just the “what.”

‍

Right-size rigor to stakes.
Save heavyweight process for high-stakes, high-uncertainty choices. Trivial decisions can run on heuristics; consequential ones need designed debate.

‍

Algorithms assist; humans are accountable.
LLMs and models expand options, but outsourcing judgment is a risk. Use machines for scale and patterning; use humans for context, values, and novel possibility.

‍

You can read the full transcript here.

‍

Timestamps

‍

00:00 Introduction to High-Stakes Decision Making

01:06 Meet the Experts: Amy Edmondson (Harvard Business School) and Mike Luca (Johns Hopkins)

02:25 The Importance of High-Quality Conversations

11:06 Case Study: Airbnb and Discrimination

13:57 Strategies for Effective Decision-Making

26:40 Pay Attention to Weak Signals

27:15 Statistical Significance and Variability

28:21 Confirmation Bias in Analytics

29:26 Generalizing Findings Across Markets

29:54 Bayesian Approach to Data Integration

32:59 Leadership and Decision-Making

36:09 The Role of Algorithms and Human Judgment

40:50 Data-Driven Leadership

46:34 Behavioral Economics and Decision Pitfalls

48:53 Concluding Thoughts and Takeaways

‍

Links From The Show

LINKS

‍

Transcript

featured

In the spotlight: Our most popular episodes

Episode

Tomasz Tunguz on Why a Trillion Dollars of Market Cap Is Up for Grabs (and How AI Teams Will Win It)

Tomasz Tunguz (Theory Ventures) joins High Signal to unpack why a trillion dollars of market cap is up for grabs as AI reshapes enterprise software. He explains why workflows are now changing faster than packaged software can keep up, how “liquid software” is redefining CRM and marketing automation, and why background agents will require a new kind of “agent inbox.” We discuss the compounding errors that arise when tools are chained too finely, the hidden AI technical debt accumulating in today’s systems, and why modular stacks—mixing local and cloud models—will beat monolithic apps. The conversation also surfaces early memory architectures, what breaks when one IC manages 100 agents, and how these shifts change the real bottlenecks in scaling AI.