Message Queues & Offline Processing with PHP

Message Queues & Offline Processing with PHP

• 1.
  UNCONMessage Queues &OfflineProcessing in PHPByMarcel EsserCode Samurai, CROSCONContact+1 202 730 9728+49 176 785 69729marcel.esser@croscon.com
• 2.
  Offline Processing ExplainedWhatis offline processing?Short answer: Running things in the background.Long answer: Offline processing is the act of queuing, prioritizing, dispatching, and executing jobs described by a message packet, received from some initiating process.Practical answer: Do the heavy stuff in a worker thread.
• 3.
  Offline Processing ExplainedNowLaterWebRequestDispatcherMessageApplicationDataStoreMessageDispatcherWorker
• 4.
  Offline Processing ExplainedNowLaterWebRequestDispatcherMessageApplicationDataStoreMessageDispatcherWorker
• 5.
  Message Queue Solutions
• 6.
  Amazon SQSProprietary WebService. 0.01USD/10,000.00 msgs + bw.Dead simple.Handles all locking.Messages up to 8kb in size.Messages expire after 4 days.Many permissions options.Automatically distributed.Runs on a 3rd party.5 functions (CreateQueue, SendMessage, ReceiveMessage, ChangeMessageVisibility, and DeleteMessage)SOAP or RESTful.
• 7.
  gearmanOpen source Cdaemon.Socket-based client libraries in several languages.PHP client library is nice and clean.Can be configured for redundancy.Fast.No Persistence. Now with minty-fresh persistence.
• 8.
  droprOpen source PHPdaemon.“trunk in production”Separate client and poll services.De-centralizedPHP-centricStorage and transport are modular. FS and cURL are implemented.
• 9.
  Apache ActiveMQOpen sourceJava daemon.Big.Fairly Java-centric.De-centralizedPHP-centricStorage and transport are modular. FS and cURL are implemented.Uses big words like ‘transport connectors’ and ‘network connectors’ and has big XML configuraton files.Just fail. Don’t even ask me.
• 10.
  beanstalkdOpen source Cdaemon.Fast.Can be persisted by writing events to a binlog.Can be distributed.Still pretty raw.Client libraries for many languages.Little bit slanted towards Rails.
• 11.
  My Homebrew (version1, PHP)Proprietary PHP library.Pretty fast.Implemented as a memcached linked list.Not even remotely difficult to write.Can scale up pretty well if done with some consideration.Used memcache for locking.I don’t feel comfortable taking PHP out of it’s web role in that way.
• 12.
  My Homebrew (version2, C)Proprietary C daemon.Very, very fast. Non-blocking epoll server. C10K and then some.Used mutexes for locking.Highly specialized for the role; not multi-purpose.Difficult to maintain.Difficult to write (properly).Difficult to deploy.
• 13.
  Zend Queue /Zend Job QueueLargely a collection of adaptors to other message queues, plus a job-based queue system.Why? Existing APIs + at command
• 14.
  Message Formats
• 15.
  Message FormatsYou needsomething other than XML and JSON because…?
• 16.
  Oh, and btw.Incase you forgot…
• 17.
  Shameless PlugHi, myname is Marcel Esser. I work for CROSCON.We do: Bespoke application developmentCustomized service monitoringMyCourt productivity softwareInfo?marcel.esser@croscon.com202.730.9728