Downloaded 34 times
Uploaded byAdam Englander
Asynchronous Python with Twisted
The document provides an overview of asynchronous programming, highlighting its advantages such as efficient CPU utilization and reduced memory footprint per connection. It introduces Twisted, an asynchronous framework for Python, detailing its deferred objects, callbacks, and error handling mechanisms. Additionally, it covers the Twisted web server features and provides examples of how to create resources and handle requests concurrently.
More Related Content
![[En] IPVS for Docker Containers](https://cdn.slidesharecdn.com/ss_thumbnails/ipvsdockerconeu2015-fullversion-151119045345-lva1-app6892-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
Similar to Asynchronous Python with Twisted
![php[tek] 2108 - Cryptography Advances in PHP 7.2](https://cdn.slidesharecdn.com/ss_thumbnails/phptek2108-cryptographyadvancesinphp7-180601213135-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
![php[tek] 2018 - Biometrics, fantastic failure point of the future](https://cdn.slidesharecdn.com/ss_thumbnails/phptek2018-biometricsfantasticfailurepointofthefuture-180601212954-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
![UiPath Automation Suite Installation (Hands-On) [2/3]](https://cdn.slidesharecdn.com/ss_thumbnails/automationsuitecommunitysession2-251015095633-a6d862f1-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
Asynchronous Python with Twisted
- 2. Who Am I? Developer/Evangelistat LaunchKey Organizer of Las Vegas Python User Group Founder/Co-Organizer of Las Vegas PHP UG Co-Organizer of Las Vegas Developer UG National Junior Basketball Coach
- 3. What is Asynchronous Programming? •Non-Linear • Event Driven • Functional • Depends on non-blocking code
- 4. Why Asynchronous? One threadfor all connections with the following benefits: Shared application level objects and data reduces memory footprint per connection Limited threads allows for better CPU utilization of each core thus fewer cores overall. Both of the above enable large applications to run on smaller virtual machines in the cloud.
- 5. What’s the Catch? Functionalnon-blocking code development has a learning curve. Developing applications with objects that persist beyond a single request can be more difficult than standard WSGI applications. Requires asynchronous libraries which can be scarce in Python.
- 6. Twisted Established Asynchronous PythonFramework 1.0 Release was in March 2004 Utilizes asynchronous I/O Server runs in Python 2.6+ and 3.2+
- 7. Deferred = Asynchronous Analogousto promises/futures Returned by asynchronous and concurrent calls Use callbacks to process result of fulfillment Process callbacks in the order they are registered with each being passed the result of the predecessor Can be quickly created for static data with defer.sucess and defer.failure functions
- 8. Callback Example d =defer.Deferred() d.addCallback(lambda x: x + “ is an “) d.addCallback(lambda x: x + “ example!“) d.callback(“This”) >>> This is an example!
- 9. Deferred Error Handling Deferredobjects catch all callback Exceptions Exceptions are wrapped in a Failure object which can also be returned by a callback or errback If no errbacks remain, exception is swallowed and processing is stopped Remaining callbacks before next errback are skipped If errback returns non-Failure, callback processing resumes
- 10. Callback Non-Failure Flow Deferredcallback(result) • callback - success • callback – success • errback - skipped • errback – skipped
- 11. Errback Non-Failure Flow Deferredcallback(result) • callback - success • errback - failure • callback – success • errback – skipped
- 12. Errback Failure Flow Deferredcallback(result) • callback - success • callback – failure • errback - failure • errback – success
- 13. Threads = Concurrent Threadsrun in thread pool managed by reactor callInThread executes function in its own thread with supplied data deferToThread works like callInThread but returns a Deferred object and processes the callbacks when complete Use threads for blocking code
- 14. Reactor Manages the mainevent loop Waits for and dispatches events or messages May have predefined services to provide access to network communications, threading, and event dispatching. Available services are operating system dependent
- 15.
- 16. Application Starts and stopsthe reactor Manages services Handles logging Can run as daemon
- 17. Service Registers with thereactor Processed in the main event loop Manages connections Communicates via Protocols
- 18. Predefined Services File Descriptor Process Threads Time(scheduling) SSL TCP UDP UNIX (UNIX sockets) Socket (3rd Party Sockets)
- 19. Protocol Utilized for interactionwith services Services with stateful connections utilize protocol factories to generate a protocol instance per connection. Provided with method to send data to connection in the form of transports or producers
- 20. Protocol Factory Builds protocolinstances Injected into every protocol instance Meant to perform inter-instance interaction Can be used as testable registry for registry pattern implementations
- 21.
- 22.
- 23. Twisted Web Web serverbuilt on Twisted HTTP Protocol Can be stand alone web server Can embedded as part of an application Supports all of the features of most modern web servers
- 24. Web Server Features Virtualhosts Proxy/Reverse proxy Scripts (hello.py) Static files CGI WSGI URL rewrites MIME processors Sessions
- 25. Simple Web Application Siteobject is root Resource Resource object represents URL segment Tree hierarchy of resources Can directly return string or write response via request object for deferred processing
- 26.
- 27. Example Hierarchy Code root= RootResource() foo = FooResource() foo.putChild(“bar”, BarResource()) foo.putChild(“bat”, BatResource()) root.putChild(“foo”, foo) root.putChild(“fiz/buzz”, FizBuzzResource()) reactor.listenTCP(8080, Site(root)) reactor.start()
- 28. Concurrent Example from twisted.webimport resource class DemoResource(resource.Resource): def render_GET(self, request): return "<html>foo</html>"
- 29. Deferred Example def render_GET(self,request): def process_resp(html, request): request.write(html) request.finish() deferred = self.async_client.request() deferred.addCallback(process_resp, request) return NOT_DONE_YET
- 30.
- 31. Example on GitHub: https://github.com/aenglander/demo-python-twisted- websocket @adam_englander #launchkeyon freenode.net #vegastech on freenode.net adam@launchkey.com