Customising Fabric Runtime and Spark in Microsoft Fabric (Day 20 of 30)
10+ hours of FREE Fabric Training: https://www.skool.com/microsoft-fabri...
Learn Apache Spark in Microsoft Fabric in the 30 days of September.
Here's the playlist for this series if you want to catchup: • Learn Apache Spark in Fabric in 30 days
Link to the GitHub for this series: https://github.com/LearnMicrosoftFabr...
Spark is the engine behind both the Data Engineering AND the Data Science experiences in Microsoft Fabric, so in September I'll be walking you through Apache Spark: what it is, why you should learn it, how to use it, and how it integrates into Microsoft Fabric.
No previous Spark knowledge is required, some basic Python would be useful!
#pyspark #microsoftfabric #apachespark
Here's the schedule:
1 Welcome - • DAY ONE - Learn Apache Spark in Micro...
2 Why Spark? • Why you should learn Spark in Microso...
3 Components of Spark - • FIVE components of the Spark engine i...
4 Spark DataFrame - • Introducing the Spark DataFrame in Mi...
5 Read Files into DataFrame - • How to read CSV, JSON, PARQUET into S...
6 Read/Write to Lakehouse Table - • How to write data to Lakehouse Table ...
7 Basic DataFrame Operations - • Learn THESE important pySpark operati...
8 DataFrame Filtering - • 14 ways to filter your Spark DataFram...
9 GroupBy and Aggregate Functions - • How to use GROUPBY and AGG functions ...
10 Handling missing values - • NO MORE NULLS - how to handle missing...
11 Joining DataFrames - • How to get better insights by JOINING...
12 Time-series data - • Analyzing DATES and TIMES in Spark in...
13 Spark SQL - • Using your SQL knowledge in Spark in ...
14-16 Spark Machine Learning - • Spark Machine Learning End-To-End Pro...
20 Configuring Spark - • Customising Fabric Runtime and Spark ...
21 Autotuning Spark Configuration - • Autotuning your Spark config in Micro...
22 Library Management - • Manage your Python packages (the old ...
23 High-concurrency mode - • Running multiple notebooks on same se...
24 Spark Scala - • Writing Spark Scala in Microsoft Fabr...
25 Fabric MSSparkUtils - • Powerful notebook utility functions (...
26 Monitoring Spark - • Monitoring Spark (errors and cost) in...
30 FINALE QnA - • How to use Spark in Microsoft Fabric ...
Timeline
0:00 Coming up...
1:21 Fabric Runtime 1.1
5:36 Ways to configure Spark
7:58 switching over to Fabric
8:35 Library Management
8:55 Configuring Spark Compute
9:05 Starter Pools
11:16 Custom Pools
14:40 Configuring Spark Compute Part 2
15:46 Configuration at Capacity level
-BROWSE MY OTHER FABRIC PLAYLISTS-
DATA ENGINEERING • Data engineering (Microsoft Fabric)
END-TO-END FABRIC PROJECT • Playlist
INTRO TO MICROSOFT FABRIC • Intro to Microsoft Fabric
DATA FACTORY • Data Factory (Microsoft Fabric)
-LINKEDIN-
Not following the LinkedIn page yet? Here's the link: / learnmicrosoftfabric
-ABOUT WILL-
Hi, I'm Will! I'm hugely passionate about data and using it to create a better world. I currently work as a Consultant, focusing on Data Strategy, Data Engineering and Business Intelligence (within the Microsoft/Azure/Fabric environment). I have previously worked as a Data Scientist. I started Learn Microsoft Fabric to share my learnings on how Microsoft Fabric works and help you build your career and build meaningful things in Fabric.
-SUBSCRIBE-
Not subscribed yet? You should! There are lots of new videos in the pipeline covering all aspects of Microsoft Fabric.
/ @learnmicrosoftfabric
Watch video Customising Fabric Runtime and Spark in Microsoft Fabric (Day 20 of 30) online without registration, duration hours minute second in high quality. This video was added by user Learn Microsoft Fabric with Will 20 September 2023, don't forget to share it with your friends and acquaintances, it has been viewed on our site 1,01 once and liked it 2 people.