Eric Colson on Why 90% of Data Science Fails—And How to Fix It
.svg)
Activation Fund
Delphina
Hugo Bowne-Anderson is an independent data and AI consultant with extensive experience in the tech industry. He is the host of the industry podcast Vanishing Gradients, a podcast exploring developments in data science and AI. Previously, Hugo served as Head of Developer Relations at Outerbounds and held roles at Coiled and DataCamp, where his work in data science education reached over 3 million learners. He has taught at Yale University, Cold Spring Harbor Laboratory, and conferences like SciPy and PyCon, and is a passionate advocate for democratizing data skills and open-source tools.
Key Quotes
Key Takeaways
1. Why Treating Data Science as a Service Function Fails
Many companies see data science as a support role, limiting it to answering business questions instead of driving strategic initiatives. The best organizations empower data scientists to generate ideas and shape decisions, not just execute predefined tasks.
2. The Hidden Value of Data Scientists: Cognitive Repertoires
Data scientists bring unique ways of framing problems—drawing from machine learning, statistics, and classic models like Markov chains and the newsvendor problem. Recognizing the right framing often matters more than the specific tools used.
3. Scaling Experimentation, Not Just Models
Most experiments fail—but that’s not a problem if you run enough of them. Organizations that embrace trial and error, rather than rigid planning, uncover the biggest wins. Data-driven companies don’t just analyze past results; they actively test their way to better decisions.
4. Decoupling Algorithms from Engineering Enables Impact
When data teams depend on engineering to deploy models, iteration slows down. Separating algorithm development from infrastructure allows data scientists to experiment rapidly, test new ideas, and drive measurable business outcomes without waiting in line for resources.
5. From Cost Center to Revenue Driver
Companies that structure data science as a core decision-making function—rather than a reporting layer—see direct business impact. The best data teams own their metrics, drive revenue, and operate autonomously, rather than waiting for permission to innovate.
You can read the full transcript here.
00:00 Data Scientist Value Left on the Table
00:28 Meet Eric Colson: DS/ML Advisor, ex-StitchFix, Netflix
01:34 The Evolving Role of Data Science
03:40 Unlocking the Potential of Data Scientists
04:10 Common Pitfalls in Data Science Utilization
07:59 The Importance of Cognitive Repertoires
09:54 Case Study: Stitch Fix's Algorithm Transformation
12:55 Leveraging Data Scientists' Unique Insights
21:30 Balancing Short-Term and Long-Term Goals
27:41 The Role of Experimentation in DS & ML
36:40 The Importance of Evidence in Data Science
37:59 Exploration and Trial Phases in DS & ML
38:29 Challenges in A/B Testing for Non-Algorithm Functions
39:36 Optionality in Data Science Experiments
45:13 Scaling Experimentation and Mitigating Risks
54:16 Organizational Changes for Data Science Impact
01:05:10 Final Thoughts and Takeaways
Links From The Show
Transcript
In the spotlight: Our most popular episodes
.png)
Gabriel Weintraub on The Hard Truth About Building AI Systems and What Most Leaders Miss About AI
In this episode of High Signal, Gabriel Weintraub (the Amman Professor of Operations, Information, and Technology at Stanford Graduate School of Business), brings his expertise in market design, data science, and operations, enriched by his experience with global platforms like Uber and Mercado Libre, to a conversation that spans practical strategies, cultural insights, and global perspectives on data and AI.
.png)
Listen up: Our latest discussions
Hear the hottest takes on data science and AI.
.png)
.png)
Get the latest episodes in your inbox
Never miss an episode of High Signal by signing up for the Delphina newsletter.