Downloaded 117 times
Uploaded byAIMDek Technologies
Introduction to Apache Kafka
Apache Kafka is an open-source distributed event streaming platform used for building real-time data pipelines and streaming apps. It was developed by LinkedIn in 2011 to solve problems with data integration and processing. Kafka uses a publish-subscribe messaging model and is designed to be fast, scalable, and durable. It allows both streaming and storage of data and acts as a central data backbone for large organizations.
More Related Content
Similar to Introduction to Apache Kafka
More from AIMDek Technologies
Introduction to Apache Kafka
- 1. Introduction to Apache Kafka By:Dhaval Thumar WWW.AIMDEK.COM
- 2. Agenda • What isApache Kafka? • How Kafka works? • History • Kafka Products • Kafka Core Components
- 3. What is ApacheKafka? ● Apache Kafka is a distributed streaming Open source platform developed by Apache Software Foundation, written in Scala and Java ○ A distributed system is a system with multiple components located on different machines that communicate and coordinate actions via messages in order to appear as a single system to the end-user. ○ Create real time data streams ○ Processing real time data stream
- 4. What is ApacheKafka?
- 5. How Kafka Works? •Kafka adopted Pub-Sub messaging architecture. Brocker Publisher Subscriber Message Broker Message Producer Message Consumer
- 6. History • Developed byLinkedin in 2011 to solve data integration problem
- 7.
- 8. Kafka Products • KafkaBroker - Central server system • Kafka Client - Producer & Consumer Java & Scala Library • Kafka Connect - Data Integration tool • Kafka Stream - Java & Scala library for Real time stream processing application • KSQL
- 9. Kafka Core Concept •Producer • Consumer • Broker/Server • Cluster • Topic • Partition • Offset • Consumer Groups
- 10.
- 11. Kafka Topic Guys, Iam collecting data from multiple producers. Which one do you want?
- 12. Kafka Topic Ok, giveme the data sent by the smart- meter aproducer
- 13. Kafka Topic Well,the smart meter producer is sending three different types of data. 1. Current-load-sent every minute 2. Consumed-units-sent every hour 3. Input-current-fluctuations-sent as and when it happens Which one do you want?
- 14. Kafka Topic Well, giveme the current-load Message
- 15. Kafka Topic Ok,so you are looking for a current-load message. That’s great! But we have a bunch of smart meter producers, and all of them are sending the current-load message. so which one of those are you interested in?
- 16.
- 17. Kafka Topic Give meall the data that you received in the current load
- 18.
- 19.
- 20. www.aimdek.com AIMDek Technologies Pvt.Ltd. +91 78747 88766 | +1 844-744-4423 Sales: sales@aimdek.com General: hello@aimdek.com AIMDek Technologies Inc. +1 647-243-6116 Sales: sales@aimdek.com General: hello@aimdek.com AIMDek Technologies Inc. +1 647-243-6116 Sales: sales@aimdek.com General: hello@aimdek.com www.aimdek.com www.aimdek.com Canada India USA