Enterprise OSGi at eBay

Enterprise OSGi at eBay

• 1.
  Enterprise  OSGi  at  eBay     Tony  Ng   Director  of  Engineering   May,  2012  
• 2.
  Agenda   •  About  eBay   •  Why  OSGi   •  How  are  we  using  OSGi   •  Lessons  learned   2  
• 3.
  $68.6 billion In merchandise soldin 2011 3  
• 4.
  eBay  at  Internet  Scale   •  100+  million  acHve  buyers  and  sellers  worldwide   •  250  million  queries  each  day  to  our  search  engine   •  300+  million  items  in  50,000  categories   •  39  countries  and  10  languages     •  9  petabytes  of  data   •  2  billion  page  views  each  day   •  75  billion  database  calls   4  
• 5.
  Challenges  for  our  Code  Base   •  ~  44  million  of  lines  of  code  and   growing   –  O(1,000)  of  jars   –  O(10,000)  of  Java  packages   knowledge complexity –  O(100,000)  of  Java  classes   •  Code  is  monolithic  -­‐  many   dependencies  and  Hght  coupling   •  Simple  code  change  ripples   throughout  the  system   code size •  Developers  slow  down  and   become  risk  averse   5  
• 6.
  Next  Gen  Pla>orm  Guiding  Principles   •  Modularity  is  first  class  design  concept   •  Technology  refresh:  prefer  open  source  technologies   •  Easy  integraHon  of  mulHple  applicaHon  frameworks   and  libraries   •  Fast  development  iteraHons   •  Team  autonomy   •  OpHmize  end-­‐to-­‐end  development  cycle   6  
• 7.
  Next  Gen  Stack   Applications (ebay.com, Mobile) E-commerce Services (Search, Item, Cart, Checkout etc) Maven / Eclipse IDE Security Resource Metadata Logging Experimentation Config Localization Tracking Data Access Svcs Fwk Orchestration JS Fwk Cloud HTML5/CSS Spring MVC JQuery JSP OSGi Spring DI Servlet Init / Handler Container (Geronimo / WAS CE) JVM Operating System (Linux) Core platform Platform frameworks Application & services
• 8.
  Our  Goals  with  Modularity   •  Tame  complexity     •  Organize  our  code  base  in  loose  coupling  fashion   •  Establish  clear  code  ownership,  boundaries  and   dependencies   •  Allow  different  components  (and  teams)  evolve  at   different  speeds   •  Increase  development  agility   8
• 9.
  Why  OSGi   Goal   How  OSGi  Helps   Hide  implementaEon  details   Export-­‐Package   Accurate  understanding  and  enforcement  of   Bundle  manifest   dependencies   (Imports  &  Exports)   SemanEc  versioning   Faster  iteraEon  for  code  change   Update  and  restart   affected  bundles  only   Isolate  and  run  mulEple  versions  of  the  same   Support  for  mulEple   modules   versions   Technology  maturity   Robust  community  and   support   9  
• 10.
  Top  Level  ConsideraEons   •  What  OSGi  features  to  use   –  Focus  on  modularity   •  End-­‐to-­‐end  development  lifecycle   –  IDE  and  non-­‐IDE  paths   –  Build,  repository  and  tool  integraHon   •  ExisHng  code  migraHon   •  Developer  learning  curve   10  
• 11.
  Bundle  Guidelines   • Create  from  boaom  up   •  Sound  architecture  boundary   •  Free  of  split  packages   •  SemanHc  versioning   •  Export  package  versions  should  match  bundle  version   •  Use  Import-­‐Package  rather  than  Require-­‐Bundle   –  With  sensible  version  ranges   •  Address  dynamic  classloading   11  
• 12.
  Dynamic  Classloading   • ProblemaHc  usage   –  Class.forName()   –  ClassLoader  loadClass  or  getResource   –  Thread  context  class  loader   •  Short-­‐term  fix   –  DynamicImport-­‐Package:  *   •  Long-­‐term  fix   –  Refactor  code  (if  possible)  so  client  can  pass  class  or   classloader  to  framework   12  
• 13.
  Bundle  CreaEon   POM Maven Bundle (Jar) with Global OSGi Maven Bundle Manifest Repository Java source Plugin files 13  
• 14.
  OSGi  Manifest  GeneraEon   •  Maven  arHfact  version  ==  bundle  version  ==  export   packages  version   •  Follow  semanHc  versioning  (semver.org)   •  Scan  code  (BND)  to  derive  import  and  export  packages   •  Use  major  version  matching  for  Import-­‐Packages:   –  Ex:  [1.1.2,2.0)   •  By  default,  all  packages  are  exported  except   –  impl.*  and  internal.*   14  
• 15.
  Two-­‐Stage  Dependency  Wiring   Maven Repo OSGi wiring A.jar; v1 Maven A.jar; v1 based on A.jar; v2 A.jar; v2 bundle B.jar; v1 B.jar; v2 subset B.jar; v2 POM C.jar; v3 Dependency a.*; v2 . b.*; v2 . . (all bundles, all Bundle Subset Runtime versions) •  Benefits   –  Smaller  bundle  set   –  IsolaHon  from  changes  in  global  repository  (reproducibility)   15  
• 16.
  App  Development  Environment   •  Eclipse  Helios   –  Geronimo  Eclipse  Plugin   –  Apache  Felix  Maven  Bundle  Plugin   –  OSGi  tooling  plugin   –  Standard  plugins  for  Maven  and  Git  integraHon   •  Others   –  Maven  archetypes  for  app  /  bundle  project  creaHon   –  Jenkins  CI   16  
• 17.
  ApplicaEon  Development   • ApplicaHons  packaged  as  EBA   •  Build  Hme:  Plajorm  parent  POM   –  Pre-­‐defined  plajorm  bundles  dependencies.  Updated   regularly.   –  App  developers  can  extend  /  override   •  RaHonale   –  CerHficaHon  across  the  specific  versions   –  Ensure  applicaHon  progresses  towards  newer  versions   –  SHll  maintain  ability  to  override  on  a  per  app  basis   17  
• 18.
  ApplicaEon  Deployment   EBA ApplicationWAR VM / Physical All dependent bundles Host Local OBR xml Activation scripts QA Cloud Deployment Configuration Files Infrastructure VM / Physical Host Container JVM VM / Physical Host App Deployment Unit Production 18  
• 19.
  ExisEng  Code  MigraEon   •  Incrementally  migrate  applicaHons  to  new  stack   •  Most  efforts  spent  in  migraHng  exisHng  library  jars   •  Built  tool  to  detect  duplicate  class  /  split  packages  across   jars   –  Merge  the  jars   –  Manual  refactoring   •  Deal  with  class  loading  code   •  Ensure  transiHve  dependencies  are  “bundle-­‐ized”   •  Need  to  migrate  third-­‐party  libraries  if  they  are  not  OSGi-­‐ enabled   19  
• 20.
  Issues  We  Encountered   •  Steep  learning  curve  for  developers   •  Build  tools  integraHon   •  Debugging  resoluHon  problems   •  IniHalizaHon  ordering  of  bundles   •  ImporHng  bundle  containing  only  non-­‐Java  resources   •  Bundle  space  isolaHon  between  container  and   applicaHons     20  
• 21.
  What  Works  Well   •  OSGi  modularity  enforcement  leads  to  beaer  code   organizaHon  and  reuse   •  Explicit  versioning  and  dependency  management   •  Solid  end-­‐to-­‐end  development  experience  with   proper  tooling  integraHon   •  Bundles  working  in  both  OSGi  and  non-­‐OSGi   environments   •  Dynamic  updates   21  
• 22.
  Lessons  Learned   • Need  a  few  in-­‐house  OSGi  experts   –  Evangelism   –  Best  pracHces   –  Support   •  Reduce  direct  exposure  to  OSGi  API  for  developers   •  Important  to  prescribe  end-­‐to-­‐end  development  lifecycle   •  Some  automaHon  to  support  migraHon  of  exisHng  code   base   •  Take  small  steps   22