Managing the "End" of a Data Pipeline
Want to build a reliable, modern data architecture without the mess?
Here’s a free checklist to help you → https://bit.ly/kds-checklist
If there is such thing as the "end" of a data pipeline, it's typically a report.
But if you've ever built one of these reports you know that it's never just a simple handoff.
There's almost always some sort of back and forth.
Or future requests for adjustments.
So what I want to talk about in this video is where you decide to actually make those logic changes.
And the secondary impacts of that decision.
The two common schools of thought here are:
1 - Making changes directly inside the report
2 - Keeping changes in the database (transformation layer code)
In this video, I want to make a case for why I personally think you're better off going with option 2.
But whether or not you agree with me...
Hopefully it'll encourage you to consider what's best for your team (& company) going forward.
Enjoy!
Timestamps:
0:00 - Intro
0:44 - Visibility
3:45 - Consistency
5:15 - Control
Title & Tags:
Managing the "End" of a Data Pipeline
#kahandatasolutions #dataengineering #reportingtool
Смотрите видео Managing the "End" of a Data Pipeline онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь Kahan Data Solutions 10 Июль 2024, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 3,100 раз и оно понравилось 124 людям.