How to Build Incremental Models | dbt tutorial
Want to build a reliable, modern data architecture without the mess?
Here’s a free checklist to help you → https://bit.ly/kds-checklist
You don't need to process every record in a table, every time.
Fortunately, dbt has a great solution for this scenario with their "incremental" materialization option.
When setup properly, they can help you significantly cut costs & processing time.
This is because incremental models only process new data vs rebuilding an entire table (the default setting).
But setting up incremental models isn't always straight forward.
It requires a few steps, an understanding of underlying functionality and some customization.
All that to say, I've noticed this process trips people up and therefore they put off implementing it.
So in today's video I'd like to help you out by covering:
What incremental models in dbt are all about
Step by step how to build one
The process to add/update new data
Thank you for watching!
Timestamps:
0:00 - Intro
0:27 - What are incremental models?
1:45 - How to use is_incremental()
4:06 - Inserting new data
5:31 - Update existing data
6:57 - Handling schema changes
9:39 - Using the --full-refresh flag
Title & Tags:
How to Build Incremental Models | dbt tutorial
#kahandatasolutions #dataengineering #dbt
Смотрите видео How to Build Incremental Models | dbt tutorial онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь Kahan Data Solutions 06 Декабрь 2023, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 15,110 раз и оно понравилось 294 людям.