Demo: Spark 3 Dynamic Partition Pruning
Watch this demo on dynamic partition pruning. Spark takes any query and converts the query into logical plan and later physical plan of the query. Dynamic partition pruning reduces the scanned data limiting it to only needed data and avoids scanning or reading of unwanted data. Using this feature will reduce the number of I/O reads and increase the speed and time of queries. This optimization is applied at run time and works when two or more fact tables are joined to refer dimension tables. The partitions are pruned at execution time when the filters are applied on the dimension table. The examples below shows how the tables are created and later joined.
Learn more here - https://blog.cloudera.com/build-on-yo...
Watch video Demo: Spark 3 Dynamic Partition Pruning online without registration, duration hours minute second in high quality. This video was added by user Cloudera, Inc. 11 September 2020, don't forget to share it with your friends and acquaintances, it has been viewed on our site 890 once and liked it 5 people.