Succeding with the Apache SOA stack

Succeeding with the Apache SOA stack
And debunking some SOA myths
Johan Edstrom
SOA Executive and Apache developer.
jedstrom@ savoirtech.com
joed@ apache.org

Jeff Genender
Principal
jgenender@ savoirtech.com

Agenda

 What is an ESB?
• Where would an “ESB” fit in?
• How do you choose the ESB?
• Reasons for picking the Apache ServiceMix stack
 Modular
 Stable
 Cluster capable

 How to design software and the build system for an ESB
• Karaf and ServiceMix are OSGi based, ServiceMix also can handle JBI.
• The Apache projects are modularized using Maven.
 Get over it and accept it :)

Key parts to scalability

 The right resources in the right place

 Messaging
• Or any other asynchronous system

 State driven

 Conversational

 Effectively aggregating

 Transactionally safe

 Correctly utilizing EE

W is an ESB?
hat

 In my world it is simply a runtime for deploying asynchronous,
event driven code.
• So, yes; of course most of you have deployed and built ESB solutions.
• Yes, of course we are partial to “ESB” solutions.

 What is an ESB in marketing slides?
• Components
• Data-transformations
• Scalability and performance (You can randomly wave your hands here)

• A Bus
 This is a mythical, magical, illusionary fantastically awesome thing that
transparently just makes everything work.

Where does the marketing ESB fit in?

 Picking up a file from a file area, transforming it with XSLT,
publishing to a web-service.

 Consuming a web-service, using a RoutingSlip and sending to a
product, then inventory queue on a JMS provider

 Consuming a web-service payload and transforming the data to
an insert in a JDBC component

 Providing “Enterprise routing infrastructure”

 Providing “Enterprise orchestration”

And what about a real world one?

 Where you need to do any of the marketing ESB stuff
• Of course they are valid use-cases

 Where you want web-apps to be able to share resources
• ServiceMix / Karaf happily will work as a servlet container

 Long running processing, “Orchestration”

 Any type of data transformation scenarios

Real one

 Multi-Protocol input/output

 Complex publish subscribe systems

 Where you need to process in parallel

 In Springism’s - anywhere you have a Bean-Ref you could start
breaking things apart.

This is what we are trying to avoid....

And this is probably what you are looking at

I’ll interpret that for y’all!

So how is that “ESB” designed?

 This is not far from an EE container
• Add OpenEJB if you want and you’ll have a full stack

 It contains a web-services layer

 Dependency injection services

 OSGi integration and legacy (JBI) support

 Camel for routing, transformation and mediation

 Aries and Spring for EE support, JNDI, TX, JPA

 Web-app deployment support

So why is that so cool?

 Well.... You can treat it in the classical manner

 Or as a really kick-ass funky container

 Or - as a nice fusion of them both!

Apache ActiveMQ

 Communications platform of choice, providing high
performance, scalability, and mission-critical reliability for
distributed enterprise computing.

 Apache ActiveMQ supports based on specs:
• JMS 1.1
• J2EE 1.4 integration-related components including JDBC, JCA, and EJBs;
dependent specifications such as JTA and JNDI;
• AJAX, REST, HTTP, TCP, SSL, NIO+SSL, UDP, multicast, JGroups and JXTA
transport protocols

ActiveMQ “Instability”

 Instability... what is it?
• Most common term used by end users
• Generally refers to ActiveMQ being frozen or completely seized
 Refers to producers being unable to send message
 Refers to queues being frozen/seized or stuck messages
 Generally temporarily is fixed by the holdover from the Microsoft days
• The 3-finger salute (ctrl-alt-delete)
• Restart of ActiveMQ
 Followed by complaints that ActiveMQ is trash
• Blog by David Pollak: “ActiveMQ: not ready for prime time”
 http://goodstuff.im/activemq-not-ready-for-prime-time
• Is ActiveMQ trash or not properly written?
 Not trash...its a very properly written... its extremely powerful
• If I have these problems, what do you do? Diagnose it first ;-)

Apache ActiveMQ

 This is another presentation :)
• So you need to attend that one if you want to debate ActiveMQ.

Apache ActiveMQ - Additional functionality

 Advisory Queues and Topics are administrative channels.
• The Broker periodically posts information about the status of known destinations
• Administration focused programs can subscribe to advisory messages like any
other Topic

 Useful life-cycle and supplementary notifications

 Advisories are generated for the following:
• Starting and stopping consumers and producers
• Creating and destroying temporary destinations
• Expiring messages on topics and queues
• Sending messages from brokers to destinations with no consumers
• Starting and stopping connections

Apache ActiveMQ, Easy to scale

 Can bridge, multicast, “cluster” as well as integrate with other
JMS providers

Apache ActiveMQ - High Availability
Pro Con
Replicated No “single” point of Supports only a single
failure slave. Requires
manual restart and
synchronization to
bring back a failed
master

Shared File No “single” point of Requires a networked
failure. Can run file-system solution
multiple slaves, with locking
automatic recovery.
JDBC No “single” point of Slow
failure. Can run
multiple slaves,

Apache Camel

 A powerful rule based routing and mediation engine.

 Provides a POJO-based implementation of Enterprise
Integration Patterns using Java DSL (Domain Specific
Language) or Spring / Aries Blueprint to configure routing and
mediation rules.

 Uses generics, annotations and URIs so that it can easily work
directly with any kind of transport or messaging model such as
HTTP, JMS, JBI, SCA, MINA or CXF Bus API without
mandating a normalized message API.

 Designed as a small library which has “minimal” dependencies
for easy embedding in any Java application.

Apache Camel Features

 Camel is a Domain Specific Language (DSL) focussed on
implementing Enterprise Integration Patterns (EIPs)
• Examples: multicast, splitter, content-based router, routing slip, dynamic routers,
aggregator

 EIPs are implemented by Camel “Processors”
• Users can develop their own processors
• A processor is a fundamental building block
• Bean language and bindings exists so that not a single piece of Apache Camel Imports
will be necessary when integrating your existing code

 Camel can connect to a wide variety of integration technologies
• Examples: JMS, HTTP, FTP, SOAP, File - There are ~ 120 components
• Integrations are implemented by Camel “Components”
• Users can develop their own components

And it is a pretty healthy library!

 Camel makes application design & development easy:
• Reduces time to create integration flows
• Reduces complexity of integration flows and makes them easier to maintain
• Routes can be deployed in mainstream typical containers (OSGi, JEE, Servlet, etc.)

 Used in several commercial offerings

 Used in “competing” offerings

 About as close to an Integrations standard as you can get

 The community stays at a high level:
• 503 subscribers to the users mailing list
• 183 subscribers to the dev mailing list

 The community is active
• ~863 mails per month on users mailing list in Mar 2011 - May 2012 (+25 %)
• ~167 mails per month on the dev mailing list in Mar 2011 - May 2012 (-15 %)
• Avg. 535 commits per month in Mar 2011 - May 2012 (+5 %)

CXF

 Simple, flexible, interceptor based, standards compliant
framework for building JAX-WS and JAX-RS services and
consumers

Additional - Application development tools

 Spring-DM and Spring

 Aries Blueprint

 Google Guice

 Felix iPojo

 And so on.....

Do I need all of that?

 Nope, many solutions will need just a few things
• jaxb, camel, jms, soap, rest and perhaps jdbc
• Cut the container down to fit your needs
• We don’t need to load all of the 100+ Apache Camel components
• Pick and choose!

 Should I run that messaging solution inside the “ESB”
• Entirely up to you, let us look a little deeper at that in a sec.

 Can I test these solutions or am I stuck with System.out.println
and a remote debugger?

I’ve heard that OSGi sucks?

 Yes, It is a bit more complex to manage a full lifecycle
• On the other hand you get reliable hot-deployment and things like configuration,
service management and subscriptions out of the box.

 Yes, package level import export can be confusing
• So make sure you use packages in a nice modular way.

 Yes, it takes a while to learn the tooling
• Then did you wake up one day just knowing that WAR or EAR layout?

 Yes, you need to learn the classpath and a bit about
classloading
• Which’ll make you a better developer in the end of the day.
• Many problems in this area actually are created by applying EE classloaders to an
OSGi environment.

More suckage

 You have to write modular software!!!!
• Yes, you’ll benefit from writing modular software
 Heh :)

 Yes, some third party libraries are “Fluster-Clucks” in OSGi
• You can always re-package em’ or use something else.
• ServiceMix bundles

 Yes, logging is strange and odd
• So Pax logging is provided

 Yes, It was hard to get Hibernate working at one point in time
• And there are like a bazillion blogs about getting it working.....

W is the simplest way of addressing teh SUKZ?
hat

 Use Apache Maven.
• Just accept it.

 Designate a maven fascist
• Keep your builds nice and clean
• Physically and monetary make sure that you abuse people breaking builds
• Structure your projects as early as you can
• Don’t be afraid of using a truckload of modules
• Get familiar with the OSGi plugins, let your modules inherit base settings
 Go steal that build setup for Apache Karaf or Apache Camel!

 Use a whiteboard and perhaps do a few diagrams every now and
then?

 Other ways of 100% guaranteed suckcess is to rely only on blogs,
random advice on irc and never reading source code.

And keep this in mind...
Spring-DM... it’s a bit like Trump’s hair...
Spring-DM... it’s a bit like Trump’s hair...

Its there... it covers quite a bit...
but it’s kinda hard to recommend.
Looks great... but needs a lot of maintenance
or it becomes a big mess.

Let’s say we liked the Apache stack!

 And let us say we have a very simple and silly use-case
• We want to accept incoming SOAP requests
• We need to inspect those HTTP headers and then use them to route
• We want this to be dynamic and flexible
 We might want to add a more complex routing engine later
• We want this to be modular and possible to run across multiple systems

 This tells us we could use
• Spring or Aries Blueprint, Google Guice or plain old Java
• We will use Apache CXF for the WS part
• We will use Apache Camel for connecting this and routing
• We will use JMS via Apache Camel-JMS

Proof of concept outline

 Let us use some EIP pattern symbols to outline this
• We haven’t really made any technology statements here
 We are going to use Apache ActiveMQ because I said so.
 It could easily have been any other JMS provider.
• We know we’ll have a synchronous part
 Web-services by nature are.
• We’ll try and make all the other parts asynchronous
• We’ll also make sure that the participants don’t need strong integration or visibility
of each other, thus we’ll (hopefully) end up with some pretty re-usable modules
• Another good approach is to also use sequence diagrams

Let’s craft some code!

 We’ll start with the Maven setup
• Use properties
• Define a parent
• Like said previously, steal a good setup
• Inherit, inherit, inherit
• Use modules

 We also ripped a WSDL from Apache CXF
• SOAP Entry point
• Gives something to code-gen against
• Common use-case

Maven

 Complicated parent pom

 Very simple sub-modules

W else do we want here?
hat

 A “Dynamic” Router!
• We’ll craft a simple header based Dynamic router from eip patterns.
 Technically it is in between a routing slip with termination and a real control
channel pattern, we utilize JMS as our control channel for this.
 We use this example to play with destinations from SoapUI

The “Dynamic Router”

 Yep, it is just a Java class.
• Imagine how easily you could integrate this with.... Anything!

W build that type of a route?
hy
 We are able to turn “endpoints”, “routing”, “systems” or whatever
you want to call it into simple destination names.

 As we get more complex we only need to control the hop-
building.

 We can break all these modules apart

So let’s sum up what we have at this point

 We defined a CXF (JAX-WS) Endpoint, generated stubs and
tied it into Camel
• The MEP (Message Exchange Pattern) is IN/OUT since we use CXF

 We then dropped in a router “Our Dynamic Router”
• It’ll listen to a JMS queue called recognizer
• Then it’ll start inspecting headers for a location

Header inspections

 A simple processor, if we wanted to we could make similar
transformations in XML, Simple, Scala and so on.

 We are just modifying the message header info.

W about that testing?
hat

 We are going to be dealing with an OSGi environment.
• Like an EE container it makes testing a tad less fun....

 Full blown container testing with something like SoapUI
• Can be scripted and automated in Maven

 Full OSGi tests with Pax-Exam
• Allows you to wire up a full container

 Simple OSGi registry tests with PojoSr
• Let’s you simulate an environment so you can test BluePrint!

 Functional Junit/TestNG tests with Camel-Test.
• Should be combined with at least one of the above!

Let’s start with a Camel-T T
est est.

 Several flavors
• Junit, Junit4, TestNG, Spring and so on.
• Exposes easy access to context, components and inlined route-builders

And the inlined route in the test

A pax exam based test - Straight from Camel’s source

Pax-Exam

 Extend the Camel Tests

 Will be fairly slow, newer Pax-Exam is improved

 Fires up a complete container

 Use this in profiles so you can “turn it off”

PojoSr ; Extend Camels Blueprint T Support
est

 Simple route, pretty much the same syntax as Spring

 This is available from Apache Camel 2.10

Okay back to our code :)
,

W got slightly distracted in looking at T Options...
e est

 First of, let us just do a Camel Test, we want to verify that the
Dynamic routing works as expected.
• As you can see, none of these routes know about each other.

Then we want to check that we can load the WS

 Blueprint context - We rely on some AMQ Magic as well!

And our wiring “Test”

 First off, the “Dummy Response”
• We simulate the real route that only grabs a message and dumps it on a queue,
we add 2 things.
 Response building
 A Mock destination for validation.

T method
est

 With this we have a full integrations test including
• JMS, CXF, OSGi ServiceRegistry, SPI lookup, Component Discovery, JAXB
validation and schema generation....

Green Bars!!!

 We’ve tested most functionality, all we really need to do
• Write the “response-builder”
• Make that response builder a bit more dynamic so it can work
 Against several queues
 With different responses
 Yes, it is a dummy class. But since it is a Java Object it is pretty darn easy to let
your imagination run free.

Dynamic config data in Blueprint

 Works like springs property placeholder

 Allows us to set defaults

 A persistent-id corresponds to a config file in ./etc in ServiceMix
or Karaf.

How we’ll be using this dynamic data

 We basically can use this to write completely dynamic steps that
will restart on a configuration change from
• File changes
• Code changes
• JMX changes

Proof of concept outline

 We now have the following modules
• customer-service : A model library
• customer-service-ws : The webservices endpoint
• dynamic-routing : A generic module for routing
• response-builder : A simple module that returns a payload

• That means that all of the needed patterns are fulfilled.
• Competing consumer is a natural thing in JMS
• Eventing is a natural part of Apache Camel
• Request Reply is “Handled for us” based on endpoint and MEP

T sum everything up
o

 Simple OSGi build, we added exports in one place.

 Completely modular design

 Re-usable bundles

 We can deploy this on one or many machines

 We can “cluster” a network of brokers

 We can easily change transport protocols

 We are covered by tests

Thank you very much for your attention!

 Questions?

 Links
• http://camel.apache.org/
• http://servicemix.apache.org/
• http://cxf.apache.org/
• http://activemq.apache.org/
• http://karaf.apache.org/

Succeding with the Apache SOA stack

More Related Content

What's hot

Similar to Succeding with the Apache SOA stack

Recently uploaded

Succeding with the Apache SOA stack