How to Create a Dynamic Pipeline in Azure Synapse Analytics or Azure Data Factory!!
@SQLBalls walks us through how to create a dynamic pipeline in @AzureSynapse . This concept could also be used for @MSDataFactory , because they are so similar. He does this to move data from a database into @AzureDataLake in parquet files so he can use an @AzureSynapse Serverless SQL Pool to query his data.
Creating a Synapse workspace
https://learn.microsoft.com/en-us/azu...
Expressions and functions in Azure Data Factory and Azure Synapse Analytics
https://learn.microsoft.com/en-us/azu...
Pipeline execution and triggers in Azure Data Factory or Azure Synapse Analytics
https://learn.microsoft.com/en-us/azu...
Chapters
00:00 Introduction
00:42 Demo - Create Linked service
01:45 Demo - Add Azure Synapse System Assigned Managed Identity to Azure SQL DB
02:43 Demo - Build Pipeline
02:50 Demo - Dynamic Lookup Task
04:03 Demo - Dynamic ForEach
04:35 Demo - Dynamic Copy Data Task
08:43 Demo - Commit & Debug the Pipeline
09:12 Demo - Validate parquet output & Query with Serverless SQL Pool
09:40 DON'T FORGET TO UNCHECK THIS BOX!
/*******************
/CONNECT WITH US!!
/*******************
Tales from the Field
Twitter: / talesftfield
Linkedin: / 100091910
Blusky: https://bsky.app/profile/talesfromthe...
Bradley Ball
Twitter: / sqlballs
Linkedin: / sqlballs
Blusky : https://bsky.app/profile/sqlballs.bsk...
Daniel Taylor
Twitter: / dbabulldog
Linkedin: / danieltaylor3
Blusky: https://bsky.app/profile/dbabulldog.b...
Neeraj Jhaveri
Twitter: / neeraj_jhaveri
Linkedin: / neerajjhaveri
Blusky: https://bsky.app/profile/neerajny.bsk...
Josh Luedeman
Twitter: / joshluedeman
Linkedin: / joshluedeman
Blusky: https://bsky.app/profile/joshluedeman...
Andrés Padilla
Twitter: / nodestreamio
Linkedin: / andrespadillaandrade
Смотрите видео How to Create a Dynamic Pipeline in Azure Synapse Analytics or Azure Data Factory!! онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь Tales From The Field 03 Май 2023, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 2,005 раз и оно понравилось 47 людям.