Downloaded 360 times
![Key/Value storage
Think memcached, GET and SET operations
[key] = value
>> SET key meh
>> GET key
meh
>> SET fucounter 0
>> INCR fucounter
1
>> GET fucounter
1](https://image.slidesharecdn.com/restmq-100821153418-phpapp02/85/RestMQ-HTTP-Redis-based-Message-Queue-3-320.jpg)
![RestMQ - Status
/stats/<queue> : Queue status
$ curl http://localhost:8888/stats/test
{
"queue" : "test",
"redis": "127.0.0.1:6379 - 10 connection(s)",
"len": {"len": 1}
}
/stats : Server status
$ curl http://localhost:8888/stats/
{
"count": 4,
"queues": [ "devops:queue", "test3:queue", "test:queue", "test2:queue"],
"redis": "127.0.0.1:6379 - 10 connection(s)"
}](https://image.slidesharecdn.com/restmq-100821153418-phpapp02/85/RestMQ-HTTP-Redis-based-Message-Queue-14-320.jpg)
![Key/Value storage
Think memcached, GET and SET operations
[key] = value
>> SET key meh
>> GET key
meh
>> SET fucounter 0
>> INCR fucounter
1
>> GET fucounter
1](https://image.slidesharecdn.com/restmq-100821153418-phpapp02/75/RestMQ-HTTP-Redis-based-Message-Queue-3-2048.jpg)
![RestMQ - Status
/stats/<queue> : Queue status
$ curl http://localhost:8888/stats/test
{
"queue" : "test",
"redis": "127.0.0.1:6379 - 10 connection(s)",
"len": {"len": 1}
}
/stats : Server status
$ curl http://localhost:8888/stats/
{
"count": 4,
"queues": [ "devops:queue", "test3:queue", "test:queue", "test2:queue"],
"redis": "127.0.0.1:6379 - 10 connection(s)"
}](https://image.slidesharecdn.com/restmq-100821153418-phpapp02/75/RestMQ-HTTP-Redis-based-Message-Queue-14-2048.jpg)
Uploaded byGleicon Moraes
RestMQ - HTTP/Redis based Message Queue
RestMQ is a message queue system based on Redis that allows storing and retrieving messages through HTTP requests. It uses Redis' data structures like lists, sets, and hashes to maintain queues and messages. Messages can be added to and received from queues using RESTful endpoints. Additional features include status monitoring, queue control, and support for protocols like JSON, Comet, and WebSockets. The core functionality is language-agnostic but implementations exist in Python and Ruby.
More Related Content
Similar to RestMQ - HTTP/Redis based Message Queue
RestMQ - HTTP/Redis based Message Queue
- 1.
- 2. WHAT !? RestMQ -Message Queue based on Redis Data manipulation: HTTP GET/POST/DELETE, COMET and WebSockets No signaling or specific protocol (optional JSON protocol) http://www.restmq.com/ Gleicon Moraes http://github.com/gleicon http://zenmachine.wordpress.com http://www.7co.cc
- 3. Key/Value storage Think memcached,GET and SET operations [key] = value >> SET key meh >> GET key meh >> SET fucounter 0 >> INCR fucounter 1 >> GET fucounter 1
- 4. Redis Key-Value database Atomic operations Semi-persistent or persistent storage Publish/Subscription channels Different Data types: Sets, Sorted Sets, Lists, Hashes http://code.google.com/p/redis/ http://github.com/antirez/redis http://rediscookbook.org/
- 5. Redis Data types Strings Lists Sets and Ordered Sets Hashes Publish/Subscribe channels
- 6. Redis Things you cando: Cache Inverted indexes Fast counters Throttle control Cooler stuff at Redis Cookbook Load Balancing Things you can't do: Search inside all values for a given string
- 7. Redis Things you can'tdo: - Search inside all values for a given key - Search inside all values for a given key - Search inside all values for a given key
- 8. Message Queues Doeseverything really needs to be tightly coupled ? Order: first come, first served Transactions: there is no transaction, just tracking Job Scheduling Publish/Subscribe Map/Reduce
- 9. Message Queues SMTP:Oldest Message Queue around Avoid hitting your DB real time with MQ Real time: Ratings, Voting and Comments Twitter Do I need NoSQL or I really need to clean my mess ?
- 10. RestMQ - EndpointsDiagram
- 11. RestMQ - Messages /q/<queue> : Queue handling (GET/POST/DELETE) $ curl -X POST -d "value=foobar" http://localhost:8888/q/testq testq:1 $ curl -X POST -d "value=foobar" http://localhost:8888/q/testq testq:2 $ curl http://localhost:8888/q/testq { "count": 0, "value": "foobar", "key": "testq:1" }
- 12. RestMQ - Messages /c/<queue> : Comet Endpoint $ curl http://localhost:8888/c/testq (hangs until there is a message to be received) /ws/<queue> : WebSocket Endpoint Needs the proper javascript code
- 13. RestMQ - Messages JSONProtocol inspired by Amazon SQS First prototype at http://jsonqueue.appspot.com add get { { "cmd" : "add", "cmd" : "get", "queue" : "testq", "queue" : "testq", "value" : "abacab" } } del take { { "cmd" : "del", "cmd" : "take", "queue" : "testq", "queue" : "testq", "key" : "testq:31" } }
- 14. RestMQ - Status /stats/<queue> : Queue status $ curl http://localhost:8888/stats/test { "queue" : "test", "redis": "127.0.0.1:6379 - 10 connection(s)", "len": {"len": 1} } /stats : Server status $ curl http://localhost:8888/stats/ { "count": 4, "queues": [ "devops:queue", "test3:queue", "test:queue", "test2:queue"], "redis": "127.0.0.1:6379 - 10 connection(s)" }
- 15. RestMQ - Controls /p/<queue> : Distribution Policy for COMET and WS Round Robin a message for each consumer Broadcast a message to all consumers /control/<queue> : Queue start/stop Pause all COMET consumers (useful for job schedulers)
- 16. RestMQ - Datastructure Diagram
- 17. RestMQ - Algorithm Algorithmfor pushing messages into a queue named 'testq' SADD testq into a SET called queueset INCR a per-queue UUID generator called testq:uuid SET testq:<<id>>, message LPUSH testq:<<id>> in to a LIST named testq:queue Works for new and existing queues. On-the-fly queue creation.
- 18. RestMQ - Algorithm Algorithmfor reading messages from a queue named 'testq' RPOP a key from a LIST named testq:queue GET the value associated to this key For non-destructive GET operations: LINDEX -1 to get a key in testq:queue LIST INCR key:refcount +1 GET the value stored for key
- 19. RestMQ - Implementation (thecore functionality can be implemented on any language) Main branch 'Enterprise Edition' Python Twisted Cyclone Redis async client Embedded version Sinatra + Ruby Reduced endpoints (GET/POST)
- 20. RestMQ - SinatraImplementation http://gist.github.com/524240 GET /q - List all queues
- 21. RestMQ - SinatraImplementation http://gist.github.com/524240 GET /q/<queue> - Get a message from <queue>
- 22. RestMQ - SinatraImplementation http://gist.github.com/524240 POST /q/<queue> - Insert a message in <queue>
- 23.
- 24.
- 25.