Netflix Data Engineering Tech Talks - Building Reliable Data Pipelines
Holden Karau, OSS Engineer, Data Platform Engineering, talks about the importance of reliable data pipelines and how to build them covering tools from testing to validation and auditing. The talk uses Apache Spark as an example, but the concepts generalize regardless of your specific tools.
Some related projects include:
https://github.com/holdenk/spark-test...
https://github.com/unionai-oss/pandera
https://github.com/target/data-validator
and
https://github.com/tensorflow/data-va....
#netflix
#datascience
#dataengineering
#etl
#bigdata
Watch video Netflix Data Engineering Tech Talks - Building Reliable Data Pipelines online without registration, duration hours minute second in high quality. This video was added by user Netflix Data 14 December 2023, don't forget to share it with your friends and acquaintances, it has been viewed on our site 8,29 once and liked it 16 people.