From Zero to Hero with Kafka Connect
Integrating Apache Kafka with other systems in a reliable and scalable way is often a key part of a streaming platform. Fortunately, Apache Kafka includes the Connect API that enables streaming integration both in and out of Kafka. Like any technology, understanding its architecture and deployment patterns is key to successful use, as is knowing where to go looking when things aren't working.
This talk covers:
* key design concepts within Kafka Connect
* Deployment modes
* Live demo
* Diagnosing and resolving common issues encountered with Kafka Connect.
* Single Message Transforms
* Deployment of Kafka Connect in containers.
📔 Slides: https://rmoff.dev/kafka-connect-zero-...
👾 Code: https://github.com/confluentinc/demo-...
⏱ Time codes:
00:00 What is Kafka Connect?
03:38 Demo streaming data from MySQL into Elasticsearch
11:43 Configuring Kafka Connect
12:33 👉 Connector plugins
13:33 👉 Converters
13:53 👉 Serialisation and Schemas (Avro, Protobuf, JSON Schema)
17:13 👉 Single Message Transforms
19:43 👉 Confluent Hub
19:51 Running Kafka Connect
20:24 👉 Connectors and Tasks
21:29 👉 Workers
21:56 👉 Standalone Worker
22:50 👉 Distributed Worker
23:10 👉 Scaling Kafka Connect
24:42 Kafka Connect on Docker
26:17 Troubleshooting Kafka Connect
27:56 👉 Dynamic Log levels in Kafka Connect
28:48 👉 Error handling and Dead Letter Queues
32:16 Monitoring Kafka Connect
32:59 Recap & Resources
--
☁️ Confluent Cloud ☁️
Confluent Cloud is a managed Apache Kafka and Confluent Platform service. It scales to zero and lets you get started with Apache Kafka at the click of a mouse. You can signup at https://confluent.cloud/signup?utm_so... and use code 60DEVADV for $60 towards your bill (small print: https://www.confluent.io/confluent-cl...)
--
💾Download Confluent Platform: https://www.confluent.io/download/?ut...
📺 Kafka Connect connector deep-dives: • Kafka Connect
✍️Kafka Connect documentation: https://docs.confluent.io/current/con...
🧩Confluent Hub: https://www.confluent.io/hub/?utm_sou...
Смотрите видео From Zero to Hero with Kafka Connect онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь Robin Moffatt 23 Октябрь 2020, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 28,82 раз и оно понравилось 46 людям.