Big Data Engineer Live Mock Interview | Topics: Pyspark, Delta Lake, Data Profiling, Data Governance

Published: 24 May 2024
on channel: Sumit Mittal
47,360
735

𝐓𝐨 𝐞𝐧𝐡𝐚𝐧𝐜𝐞 𝐲𝐨𝐮𝐫 𝐜𝐚𝐫𝐞𝐞𝐫 𝐚𝐬 𝐚 𝐂𝐥𝐨𝐮𝐝 𝐃𝐚𝐭𝐚 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫, 𝐂𝐡𝐞𝐜𝐤 https://trendytech.in/?src=youtube&su... for curated courses developed by me.

I have trained over 20,000+ professionals in the field of Data Engineering in the last 5 years.

𝐖𝐚𝐧𝐭 𝐭𝐨 𝐌𝐚𝐬𝐭𝐞𝐫 𝐒𝐐𝐋? 𝐋𝐞𝐚𝐫𝐧 𝐒𝐐𝐋 𝐭𝐡𝐞 𝐫𝐢𝐠𝐡𝐭 𝐰𝐚𝐲 𝐭𝐡𝐫𝐨𝐮𝐠𝐡 𝐭𝐡𝐞 𝐦𝐨𝐬𝐭 𝐬𝐨𝐮𝐠𝐡𝐭 𝐚𝐟𝐭𝐞𝐫 𝐜𝐨𝐮𝐫𝐬𝐞 - 𝐒𝐐𝐋 𝐂𝐡𝐚𝐦𝐩𝐢𝐨𝐧𝐬 𝐏𝐫𝐨𝐠𝐫𝐚𝐦!

"𝐀 8 𝐰𝐞𝐞𝐤 𝐏𝐫𝐨𝐠𝐫𝐚𝐦 𝐝𝐞𝐬𝐢𝐠𝐧𝐞𝐝 𝐭𝐨 𝐡𝐞𝐥𝐩 𝐲𝐨𝐮 𝐜𝐫𝐚𝐜𝐤 𝐭𝐡𝐞 𝐢𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰𝐬 𝐨𝐟 𝐭𝐨𝐩 𝐩𝐫𝐨𝐝𝐮𝐜𝐭 𝐛𝐚𝐬𝐞𝐝 𝐜𝐨𝐦𝐩𝐚𝐧𝐢𝐞𝐬 𝐛𝐲 𝐝𝐞𝐯𝐞𝐥𝐨𝐩𝐢𝐧𝐠 𝐚 𝐭𝐡𝐨𝐮𝐠𝐡𝐭 𝐩𝐫𝐨𝐜𝐞𝐬𝐬 𝐚𝐧𝐝 𝐚𝐧 𝐚𝐩𝐩𝐫𝐨𝐚𝐜𝐡 𝐭𝐨 𝐬𝐨𝐥𝐯𝐞 𝐚𝐧 𝐮𝐧𝐬𝐞𝐞𝐧 𝐏𝐫𝐨𝐛𝐥𝐞𝐦."

𝐇𝐞𝐫𝐞 𝐢𝐬 𝐡𝐨𝐰 𝐲𝐨𝐮 𝐜𝐚𝐧 𝐫𝐞𝐠𝐢𝐬𝐭𝐞𝐫 𝐟𝐨𝐫 𝐭𝐡𝐞 𝐏𝐫𝐨𝐠𝐫𝐚𝐦 -
𝐑𝐞𝐠𝐢𝐬𝐭𝐫𝐚𝐭𝐢𝐨𝐧 𝐋𝐢𝐧𝐤 (𝐂𝐨𝐮𝐫𝐬𝐞 𝐀𝐜𝐜𝐞𝐬𝐬 𝐟𝐫𝐨𝐦 𝐈𝐧𝐝𝐢𝐚) : https://rzp.io/l/SQLINR
𝐑𝐞𝐠𝐢𝐬𝐭𝐫𝐚𝐭𝐢𝐨𝐧 𝐋𝐢𝐧𝐤 (𝐂𝐨𝐮𝐫𝐬𝐞 𝐀𝐜𝐜𝐞𝐬𝐬 𝐟𝐫𝐨𝐦 𝐨𝐮𝐭𝐬𝐢𝐝𝐞 𝐈𝐧𝐝𝐢𝐚) : https://rzp.io/l/SQLUSD

BIG DATA INTERVIEW SERIES

This mock interview series is launched as a community initiative under Data Engineers Club aimed at aiding the community's growth and development

Our highly experienced guest interviewer, Chandrali Sarkar,   / chandrali-sarkar-4570a1102   shares invaluable insights and practical guidance drawn from her extensive expertise in the Big Data Domain.

Our expert guest interviewee, Abhirup Dey,   / abhirup3193   has an interesting approach to answering the interview questions on Apache Spark, SQL and Azure Cloud Services.

Link of Free SQL & Python series developed by me are given below -
SQL Playlist -    • SQL tutorial for everyone by Sumit Si...  
Python Playlist -    • Complete Python By Sumit Mittal Sir  

Don't miss out - Subscribe to the channel for more such informative interviews and unlock the secrets to success in this thriving field!

Social Media Links :
LinkedIn -   / bigdatabysumit  
Twitter -   / bigdatasumit  
Instagram -   / bigdatabysumit  
Student Testimonials - https://trendytech.in/#testimonials

TIMESTAMPS : Questions Discussed
0:00 Project discussion
2:57 Difference of Delta Lake and Data Lake
3:27 What is the use of Unity Catalog
4:34 What is Data Profiling ?
5:14 What is Data Goverance ?
7:30 xplain the 3 main key features of Unity Catalog?
9:00 How much size of data you are handling in your day to day project?
10:52 Explain Parquet File Format?
11:55 DataFrame Vs Dataset ? Which is better?
14:43 Lazy Evaluation in Spark?
17:37 Examples of Narrow & Wide Transformations
18:07 How can we lessen the shuffle?
19:55 Coalesce and Repartition
20:36 Steps involved after submitting the spark job?
21:22 Explain about Partitions in Spark?
22:28 Scenario based question
28:26 Deep Copy and Shallow Copy in Python
20:00 Series and Dataframes in Pandas
32:50 Python Coding Question
39:33 SQL Question 1
42:02 SQL Question 2

Music track: Retro by Chill Pulse
Source: https://freetouse.com/music
Background Music for Video (Free)

Tags
#mockinterview #bigdata #career #dataengineering #data #datascience #dataanalysis #productbasedcompanies #interviewquestions #apachespark #google #interview #faang #companies #amazon #walmart #flipkart #microsoft #azure #databricks #jobs


Watch video Big Data Engineer Live Mock Interview | Topics: Pyspark, Delta Lake, Data Profiling, Data Governance online without registration, duration hours minute second in high quality. This video was added by user Sumit Mittal 24 May 2024, don't forget to share it with your friends and acquaintances, it has been viewed on our site 47,360 once and liked it 735 people.