WebSocket MicroService vs. REST Microservice

WebSocket MicroService vs. REST Microservice

• 1.
  Rick Hightower -Mammatus Technology Inc. WebSocket MicroService vs. REST Microservice How does the performance of a REST microservice compare to a WebSocket MicorService?
• 2.
  Comparison of Speed ❖WebSocket is faster and uses less resources for more calls ❖ REST is slower due to each connection needing to wait for a response before it can send another request
• 3.
  REST/HTTP vs WebSocket ❖With REST you have to wait for a reply before you can send another request (over the same connection) ❖ Unless you use pipelining which seems to work better in benchmarks than the real world ❖ With WebSocket you can stream requests and stream responses and then marry them together ❖ Still request/reply but WebSocket can send responses while it is getting other requests
• 4.
  Example uses QBit ❖QBit supports REST and WebSocket RPC ❖ QBit allows you to stream requests over the wire and receive responses over the wire. It employs micro- batching to batch requests and responses into larger chunks to minimize IO and thread hand-off costs ❖ QBit allows async REST calls and async WebSocket calls. (As well as internal services based on queues)
• 5.
  wrk Testing toolfor REST/HTTP ❖ To perf test REST code we use wrk ❖ wrk load tester capable of generating significant load when run on a single multi-core CPU ❖ wrk load tester uses multithreaded and event notification systems ❖ epoll and kqueue ❖ wrk maximizes the number of HTTP requests per second
• 6.
  Trade Service
• 7.
  wrk LUA script
• 8.
  Caveats About theTest ❖ We can tweak server a bit and reduce the flush rate or reduce the batch size to get higher throughput with lower connections ❖ We can also tweak the OS so we can have more ephemeral ports available and then use a lot more connections ❖ experience tells me that we can get close to 90K TPS or so on a MacBook Pro ❖ We could test from two machines ❖ one of those machines being a Linux server with a tweaked stack ❖ This test has disadvantage of all being run on same machine ❖ but same disadvantage that the WebSocket version will have so it is somewhat fair ❖ We could also employ HTTP pipelining to increase the throughput, ❖ great trick for benchmarks rarely works in production environments with real clients ❖ On commodity Linux server, we can get close to 150K TPS to 200K TPS from experience
• 9.
  WebSocket Test ❖ Thiswill be a bit harder ❖ No tool like wrk to test WebSocket RPC calls so we will write our own ❖ RPC calls are QBit centric and rely on JSON plus some custom ASCII protocol ❖ Some overhead in the protocol because we have to match responses to clients and requests need unique identifier (as well as the client needing a unique identifier)
• 10.
  To use WebSocketwe will need Async interface to service
• 11.
  WebSocket PERF Testcode
• 12.
  WebSocket Perf code continued
• 13.
  How does theWebSocket microservice do? 1 Million messages a second. 700K messages for a single WebSocket connection. 7.5x better than REST version.
• 14.
  Overhead ❖ Running codethrough perf showed it was waiting for IO a lot. Better IO. Better NIC card. More TCP/IP stack tuning. You can expect better throughput. ❖ QBit has to marry requests with responses/client id combinations. There is overhead in doing this. QBit also uses JSON/ASCII protocol which could be optimized to reduce IO (gzip it or use message pack). ❖ Custom more efficient streaming can be implemented if needed (from experience, this is another 2x to 4x) ❖ But in general, WebSocket will do better than HTTP
• 15.
  More Caveats ❖ Whenusing WebSocket, you also need service discovery, and you need to implement your own failover ❖ You can’t just throw a bunch of microservices behind a load balancer ❖ QBit integrates with DNS and consul.io to provide service discovery (also allows you to use push)
• 16.
  Follow up ❖ Thisslide deck as a blog post: ❖ http://rick-hightower.blogspot.com/2015/12/websocket-microservice-vs-rest.html ❖ The QBit project: ❖ https://github.com/advantageous/qbit ❖ Mammatus Technologies ❖ http://www.mammatustech.com/ ❖ Rick Hightower ❖ https://twitter.com/RickHigh ❖ https://www.linkedin.com/in/rickhigh
• 17.
  There is alot more to QBit than this
• 18.
  QBit supports reactivestream over WebSocket
• 19.
  QBit implements Service Actors
• 20.
  Check out QBit https://github.com/advantageous/qbit