C++ scalable network_io

C++ scalable network_io

• 1.
  Scalable Network I/Owith C++ Arindam Mukherjee Pune C++ and Boost Meetup
• 2.
  Content • Sockets • BoostAsio programming model • Synchronous model • Key challenges to scaling • Asynchronous model • Asio coroutines
• 3.
  Objectives • Get afeel of the subject • Know where to look, what to look for
• 4.
  Sockets • Applications bindto an IP address and a numeric port to start communicating. • Port: an unsigned integer identifying the point of attachment of the app to the protocol. • Endpoint: The treble of an IP address, port and protocol (TCP, UDP, etc.). • Socket: an unsigned integer identifying an endpoint in use.
• 5.
  Socket API • BerkeleySockets API, part of POSIX standard. • System calls – socket: get a socket – bind: bind to an endpoint (usually passive) – listen: wait for incoming connections (passive) – accept: establish the TCP connection (passive) – connect: connect to a remote endpoint (active) – close: tear down the connection
• 6.
  Boost Asio ProgrammingModel • asio::io_service as a bridge to the OS, and as a task queue. • IO objects like asio::ip::tcp::socket, likewise for udp, etc. • I/O functions both sync and async, like asio::read/write/asycn_read. • Helper classes for name resolution, IP address manipulation.
• 7.
  Name resolution • asio::ip::tcp::resolverprovides the resolve method. – Takes asio::ip::tcp::resolver::query object. – Returns iterator of type asio::ip::tcp::resolver::iterator. – Dereferencing the iterator return asio::ip::tcp::endpoint objects.
• 8.
  Vectored I/O • Gatherdata from disparate areas in memory and write it to a connection as a single stream. • Read data from a single connection and partition the stream across a set of memory areas. • Provided by readv / writev system calls – requires lesser context switches or processing overheads.
• 9.
  Asio Buffers forvectored I/O • Wrappers around memory buffers, initialized with buffer base address and length in bytes. • asio::const_buffer – source, asio::mutable_buffer – sink. • Sequence of buffers can be vectors, lists, etc.
• 10.
  Sync I/O withTCP (server) • Create a TCP listener socket. • Bind to a local endpoint. • Listen for and accept incoming connections. • Perform I/O and handle events on each connection, using a per-connection socket.
• 11.
  Sync I/O withTCP (client) • Create a TCP socket (and optionally bind to an endpoint). • Connect to a remote endpoint (server). • Perform communication and handle events on the connection.
• 12.
  TCP sync I/Ointeraction model
• 13.
  Key challenges toscaling • Sync APIs block waiting for completion. • One thread per connection is for toy programs: scalability anti-pattern. • Use a thread-pool and leverage the io_service task queue: only limits threads, not waiting. • Go async: wait-free. • Async + thread-pool: wait-free, limited threads.
• 14.
  io_service as atask queue • Put a handler with signature void() on the queue, using post or dispatch. • Handlers associated with tasks, could be ready or pending. • Call run or run_one from threads to process tasks. • Call poll or poll_one to process ready tasks. • Create object of asio::io_service::work to make run wait. • Serialize handlers using asio::io_service::strand.
• 15.
  Async I/O • Submittasks to the OS via the io_service, and be notified by callbacks. • Wait-free, but difficult to code. • Maintaining context / state across callback invocations presents challenges.
• 16.
  Async I/O model
• 17.
  Coroutines • A coroutinecan be called, suspended and resumed. • Suspension through cooperative yielding. • On suspension, control goes back to counterpart (caller). • On resumption, coroutine does not restart but continues from where it left off. Stack preserved. • Boost.Coroutine provides implementation.
• 18.
  Async I/O withcoroutines • Enqueue a coroutine using asio::spawn. • Coroutine runs, but instead of blocking, registers the yield context as a handler, and then yields. • Control goes to caller (loop in run function). • The run function handles other I/O requests. • On I/O events, handler is invoked which resumes coroutine.
• 19.
  Coroutine Sequence diagram •We’ll make this during the presentation.
• 20.
  Q & A •Books: Learning Boost C++ Libraries (Packt), Chapter 11. • References: http://www.gamedev.net/blog/950/entry- 2249317-a-guide-to-getting-started-with- boostasio/ • Collaboration diagrams from here: http://think-async.com/Asio/asio- 1.4.1/doc/asio/overview/core/basics.html
• 21.
  Howto lambda • Syntaxfor generating function objects inline: [] { std::cout << "Hello, world!n"; } • Can take args: [](std::string name) { std::cout << "Hello, " + name + "!n"; } • Can access vars from surrounding scope: [&var1, var2](…) { std::cout << var2 << 'n'; var1 = 10;}
• 22.
  Howto bind • Syntaxfor adapting function arity (signature). void foo(int n, std::string name); // binary auto f = boost::bind(foo, ::_1, "hello"); // unary • Binding references: void foo(const std::string& name); // unary std::string str; auto f = boost::bind(foo, boost::ref(str)); // nullary • Placeholders for unbound params: ::_1, ::_2.