Integrate Schema Registry & Kafka in Python...｜蘇揮原 Mars Su｜PyCon APAC 2022
PyCon APAC 2022｜一般演講 Talks｜國泰金控 Cathay Financial Holdings / 美光科技 Micron 冠名贊助
✏️ 共筆 Note：https://hackmd.io/@pycontw/BkwCLT7Ji
🖐🏻 Slido：https://app.sli.do/event/sE3aaBEUsYZ7...
🪧 投影片 Slides：https://drive.google.com/file/d/1NdBa...
💬 語言 Language：中文演講/英文投影片 Chinese talk w. English slides
🎯 層級 Level：中階 Intermediate
🔎 分類 Category：應用 Application
💡 摘要 Abstract 💡
In the current data-driven world, we are often faced with how to process and analyze data effectively and in real time. And streaming processing will be an important application. In addition, the data will have different schemas for different applications and needs. In order to effectively achieve data correctness and availability in the application of streaming, it is necessary to integrate schema verification into the streaming process. In order to achieve this objective, I will start with introducing the concept and use cases or scenarios of streaming process and two services, Apache Kafka and Schema Registry. The Kafka is a message queue system that can handle a large amount of streaming data. And Schema Registry is a service which based on Kafka, it can help us do schema verification during producing data to Kafka or consuming data from Kafka. Lastly, I will share how to use python to integrate these two service to implement a reliable streaming process.
🚀 講者介紹 About Speaker - 蘇揮原 Mars Su 🚀
A Senior ML/Data Engineer in Gogolook. Currently i am in charge of implementing streaming etl infrastructure and nlp related ml model and application. Having 4+ years experience of data science and data engineering, include NLP and Streaming(micro-batch) ETL design. My research interests include nlp related algorithm model and paper, streaming data pipeline and cloud service. Hope i can contribute something in data world.
#python #pycontw #pyconapac2022 #kafka #streaming
Follow “PyCon Taiwan”
⭐️ Official Website: https://tw.pycon.org
⭐️ Facebook: / pycontw
⭐️ Instagram: / pycontw
⭐️ Twitter: / pycontw
⭐️ LinkedIn: / pycontw
⭐️ Blogger: https://pycontw.blogspot.com
Смотрите видео Integrate Schema Registry & Kafka in Python...｜蘇揮原 Mars Su｜PyCon APAC 2022 онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь PyCon Taiwan 11 Сентябрь 2022, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 544 раз и оно понравилось 15 людям.