The AI Paradox: Why Your Data Team’s Workload is About to Explode

Snowflake
Chris Child currently serves as the VP of Product, Data Engineering at Snowflake, where he oversees Snowflake's data engineering, open lakehouse, open source, and developer products. Since joining Snowflake in 2018, Chris has focused on product experiences. He most recently served as VP, Worldwide Sales Engineering at Snowflake, leading the company's global sales engineering team and hearing first-hand from customers how they use Snowflake's product portfolio to drive value for their businesses. Prior to joining Snowflake, Chris led product teams at a number of companies specializing in data, including Segment, Salesforce, RelateIQ, and Foursquare. Chris also spent time as both an early stage and late stage technology investor. He is an alum of MIT and Stanford and loves to be doing anything outdoors with his wife Lyndsey and three kids.

Delphina
Hugo Bowne-Anderson is an independent data and AI consultant with extensive experience in the tech industry. He is the host of the industry podcast Vanishing Gradients, a podcast exploring developments in data science and AI. Previously, Hugo served as Head of Developer Relations at Outerbounds and held roles at Coiled and DataCamp, where his work in data science education reached over 3 million learners. He has taught at Yale University, Cold Spring Harbor Laboratory, and conferences like SciPy and PyCon, and is a passionate advocate for democratizing data skills and open-source tools.
Key Quotes
Key Takeaways
AI Currently Creates More Work Than It Saves for Data Engineering.
While AI assistants offer productivity gains, they unlock vast new datasets and drive demand for new pipelines, resulting in a net increase in data engineering work. A recent MIT survey found 77% of leaders see their data engineers’ workloads growing despite AI adoption.
LLMs Fail Without a Semantic Layer.
Off-the-shelf LLMs generate poor SQL because they lack business context. Chris found the key wasn't more training data, but providing a semantic model that defines core business concepts like "customer" or "revenue" for the AI to use.
Your CIO Doesn't Get Data Engineering's Value.
While 72% of executives see data engineers as integral to the business, only 55% of CIOs do. Chris suggests this is because CIOs remain platform-focused, while business owners and Chief Data Officers see the direct impact on revenue and insights.
The Future Data Engineer Manages Agents.
The role is evolving from an individual contributor writing code to a manager of agents. This involves setting budgets, defining goals, and providing architectural oversight to ensure a portfolio of automated pipelines delivers business value without creating chaos.
AI Risk Is a Governance Problem, Not a New Problem.
Trusting AI-generated outputs and pipelines doesn't require new technology. It requires extending existing data governance frameworks (like access controls, data quality checks, and security rules) to apply to agents, ensuring they inherit the same permissions as a human would.
To Move Fast on AI, First Move Slowly on Foundations.
Companies with a solid data foundation (unified data with strong governance) are deploying production AI applications far more quickly. They can bypass foundational work and focus on building, as Snowflake did with its internal go-to-market agent.
Data Engineers Must Become Product Managers.
The future value of data engineering isn't writing code, but asking "why" to understand the underlying business problem. This shifts the role from a technical executor fulfilling pipeline requests to a strategic partner who architects solutions for business outcomes.
The Most Important Skill is Killing Pipelines.
As AI makes it easy to spin up new pipelines, the critical skill shifts from building them to rigorously calculating their ROI. Data leaders must get good at identifying and shutting down low-value experiments to avoid sprawl and wasted cost.
You can read the full transcript here.
Timestamps
00:00 Introduction to Data Engineering Challenges
01:04 The Role of Data Engineers in AI
02:09 Chris Child's Insights on AI and Data Engineering
02:14 MIT Report and Data Engineering Evolution
03:12 The Growing Demands on Data Engineering
05:29 AI's Impact on Data Engineering Workloads
07:56 The Future of Data Engineering with AI
10:55 Challenges in AI-Assisted Data Engineering
21:12 Business Leaders' Perspectives on Data Engineering
26:03 Evaluating Business Value in Data Pipelines
27:33 The Evolving Role of Data Engineers
28:17 Addressing Risks and Governance in AI
31:55 Speed vs. Quality in AI Data Applications
35:32 Organizational Changes in an AI-First World
43:28 Career Advice for Data Engineers
45:48 Making Organizations AI Ready
49:14 Conclusion and Final Thoughts
Links From The Show
- MIT Technology Review Report: Redefining Data Engineering in the Age of AI
- The Evolution of the Modern Data Engineer: From Coders to Architects
- Why Most AI Agents Fail (and What It Takes to Reach Production) with Anu Brahadwaj (Atlassian)
- The End of Programming As We Know It with Tim O'Reilly
- The Incentive Problem in Shipping AI Products — and How to Change It with Roberto Medri (Meta)
- Andrej Karpathy — AGI is still a decade away
- Chris Child on LinkedIn
- High Signal podcast
- Watch the podcast episode on YouTube
- Delphina's Newsletter
Links From The Show
Transcript
In the spotlight: Our most popular episodes
Listen up: Our latest discussions
Hear the hottest takes on data science and AI.
Get the latest episodes in your inbox
Never miss an episode of High Signal by signing up for the Delphina newsletter.


.png)









