Learn RabbitMQ with Python in 90mins

Learn RabbitMQ with Python in 90mins

• 1.
  Larry cai <larry.caiyu@gmail.com>
• 2.
  Agenda  Introduction  Exercise1: Hello World  Exercise 2: Work queues - Direct  Exercise 3: Publish/Subscribe - Fanout  Exercise 4: Routing - Direct  Exercise 5: Topics  Reference 2 05/11/15 Mostly are based https://www.rabbitmq.com/getstarted.html https://www.rabbitmq.com/
• 3.
  Environment Preparation (docker) Boot2docker Installer (127M)  Contains latest docker already, fast  Container persistence via disk automount on /var/lib/docker  $ docker -v  User/Passwd: docker/tcuser (192.168.59.103)  Download python/rabbitmq:2 docker images  $ docker pull python:2  $ docker pull rabbitmq:3-management  (Windows/Mac) Clone the code from github to your user directory (~/git) https://github.com/larrycai/codingwithme-rabbitmq  Notepad++ & MobaXterm are recommended 3 05/11/15 http://boot2docker.io/
• 4.
  Introduction  RabbitMQ isa lightweight, reliable, scalable and portable message broker. (Apache ActiveMQ/Apollo, ZeroMQ)  It supports AMQP (Advanced Message Queuing Protocol)  Written in Erlang/OTP  Messaging: Pattern (asynchronous)  Producer: create message  Consumer: Handle a message  Queue: Stack the message 4 05/11/15
• 5.
  Exercise 1: HelloWorld  Run first app inside docker environment $ cd /c/Users/<id>/git/codingwithme-rabbitmq $ docker run -d -e RABBITMQ_NODENAME=rabbit --name rabbit -p 15672:15672 rabbitmq:3-management $ docker run -v $PWD:/code -w /code --name worker --link=rabbit:rabbit -it python:2 bash # pip install pika # cd exer1 && ./send.py  Browse: http://192.168.59.103:15672 See overview and Check Exchange/Queue Get message  Run in another shell inside docker $ docker exec –it worker bash # cd exer1 && ./receive.py Check queue again in GUI  Update the “Hello world” to “Hello Larry” 5 05/11/15 Docker HostDocker Host RabbitMQ Server (rabbit) 15672 Python:2 (worker) Python:2 (worker)
• 6.
  RabbitMQ - Concept Publisher/Consumer is app (running in python container in exercise 1)  Publish & consume the messages  In RabbitMQ, An exchange receives messages from producers and pushes them to queues. 6 05/11/15
• 7.
  Rabbit Libraries -Python Pika module  Pika is RabbitMQ server python module  Procedure for Publish 1. Establish connection 2. Define Queue (“Hello”) 3. Publish (routing_key is queue name)  Consume  1,2 same for publish  3: Call back 7 05/11/15
• 8.
  Queue in depth Round-robin dispatching  Scale to distribute to different consumers  Message acknowledgment  An ack(nowledgement) is sent back from the consumer to tell RabbitMQ that a particular message had been received  Message durability & Fair dispatch (self-learning) 8 05/11/15
• 9.
  Exercise 2: Workqueues  How to scale the twitter message image processing  twitter.py – publish twitter message  image.py – process the image if twitter has  Env # cd ../exer2 #./twitter.py “I see the nice picture 1.png” #./image.py  Did we receive the message in queue ? Why ?  Run more image.py to see how it is distributed (another shell, docker exec -it .. )  Kill some image.py 9 05/11/15
• 10.
  Exchange types  Exchangetype is the rule on how to deliver message to the queues: direct, fanout, topic and headers  Direct: if the routing key matches, then the message is delivered to the corresponding queue.  Fanout: multicast the received message to the bound queues.  Publish/Subscribe Patten  Broadcast 10 05/11/15
• 11.
  Exercise 3: Fanout- Publish/Subscribe  How to broadcast the message for different consumers  twitter.py – publish twitter message  image.py – manipulate the image if twitter has  Env # cd ../exer3 #./twitter.py “I see the nice picture 1.png” #./image.py  Did we receive first twitter & Check in the browser for the queue  Implement extra tag.py check whether twitter has tag  Define queue `tag` and bind to `twitter` exchange  Search for “#xxx#” keywords and print it out  ./twitter.py “#codingwithme# is good, send picture 1.png” 11 05/11/15
• 12.
  Binding  A bindingis a relationship between an exchange and a queue.   One to one mapping  Multiple mapping with Routing_key 12 05/11/15
• 13.
  Exercise 4: Direct- Routing  How to handle different log system  Env # cd ../exer4 # ./emit_log_direct.py # ./receive_logs_direct.py  Exercise:  Check GUI for queue name and bindings  More receiver # ./receive_logs_direct.py warning error > logs_from_rabbit.log # ./receive_logs_direct.py info warning error  Define fixed queue name 13 05/11/15
• 14.
  Topics  User patternas routing key (“a.b.c.d”)  Queues can use wildcard characters in binding  * matches a single word, # zero or more words.  “rabbit.*” matches for rabbit.info/rabbit.error  Topic exchange is powerful and can behave like other exchanges.  When a queue is bound with "#" (hash) binding key - it will receive all the messages, regardless of the routing key - like in fanout exchange.  When special characters "*" (star) and "#" (hash) aren't used in bindings, the topic exchange will behave just like a direct one. 14 05/11/15
• 15.
  Exercise 5: Topic handle different log system more  Env # cd ../exer5 # ./emit_log_direct.py # ./receive_logs_direct.py “#”  Exercise  Check the queue name in GUI  Create for rabbit.*/apache.*/*.info  Publish message for rabbit.info/error & apache.info/error 15 05/11/15
• 16.
  Summary  RabbitMQ isthe message broker support AMQP  Handle async messages  AMQP Model  Different Exchange types  Messages Queues  Bindings  Rule for Binding .. 16 05/11/15
• 17.
  Reference  http://www.rabbitmq.com/  Exercisesare based on http://www.rabbitmq.com/getstarted.html  Book:  RabbitMQ in Action (2012): http://www.manning.com/videla/  Code:  https://github.com/larrycai/codingwithme-rabbitmq 17 05/11/15
• 18.
  ChangeLog  2015/05/11: firstrelease 18 05/11/15