Managing the "End" of a Data Pipeline
Want to build a reliable, modern data architecture without the mess?
Here’s a free checklist to help you → https://bit.ly/kds-checklist
If there is such thing as the "end" of a data pipeline, it's typically a report.
But if you've ever built one of these reports you know that it's never just a simple handoff.
There's almost always some sort of back and forth.
Or future requests for adjustments.
So what I want to talk about in this video is where you decide to actually make those logic changes.
And the secondary impacts of that decision.
The two common schools of thought here are:
1 - Making changes directly inside the report
2 - Keeping changes in the database (transformation layer code)
In this video, I want to make a case for why I personally think you're better off going with option 2.
But whether or not you agree with me...
Hopefully it'll encourage you to consider what's best for your team (& company) going forward.
Enjoy!
Timestamps:
0:00 - Intro
0:44 - Visibility
3:45 - Consistency
5:15 - Control
Title & Tags:
Managing the "End" of a Data Pipeline
#kahandatasolutions #dataengineering #reportingtool
Watch video Managing the "End" of a Data Pipeline online without registration, duration hours minute second in high quality. This video was added by user Kahan Data Solutions 10 July 2024, don't forget to share it with your friends and acquaintances, it has been viewed on our site 3,100 once and liked it 124 people.