Ride the Camel

A very, very brief introduction Apache Camel.

Posted: Wed 16 Mar, 2016, 17:40
Camel

Not a real camel, Apache Camel. Although I am sure real camel riding is also great fun.

Apache Camel has, without over-stating, revolutionised the work I have done on two recent projects. Camel allows developers to build a system based on Enterprise Integration Patterns (EIP). EIP is a long established approach that tackles common programming problems as a workflow/routing exercise rather than a series of sequential method operations. All problems are re-stated in terms of the Splitters, Filters, Aggregators etc and a workflow is defined. This provides many important benefits:

Integration is Done. Focus on the business.

Integration with JDBC, Files, JMS, RabbitMQ, AWS or MINA is extremely simple and all follow a common pattern. This means the underlying transport, connectivity and protocols is all taken care of with tried-and-tested code, leaving you to focus on the business requirement. Moving from MINA to Jetty is as simple as changing a URL giving full abstraction from these underlying concerns.

Parallelism

Once the flow is defined using a RouteBuilder it can be carved up into separarate asynchonous sections. Whilst it is possible to do one large synchronous sequence of processing (appropriate for a JMS transaction) it is also very easy to use SEDA queues to de-couple sections and allow them to proceed separately. With this in place, consumer count can simply be increased to parallelise.

Example: Reading from a file and writing to a SEDA queue

from("stream:file?fileName=${infile}")
            .process { row ++ }
            .to("seda:filestream?size=3000&blockWhenFull=true")
        

Freestyle DSL for Configuration

Writing configuration in XML is great for many cases. However writing configuration in a DSL is so much better for making readable configuration and leveraging all the standard benefits of using a language e.g. method suggestion. For example it is simple with the Groovy DSL to embed code in one of the pipeline of actions rather than having to refer to some other Bean all the time (not possible with Java DSL). This make the code much quicker to develop and infinitely more readable. Likewise it is easier to leverage boolean logic for conditionals with the DSL which always look very clunky in XML.

Example: A timer to periodically retrieve a list of RSS/Atom sources

	from("quartz2://rsstime?trigger.repeatInterval=120000&trigger.repeatCount=-1")
        		.split { Exchange exchange ->
        			def feeds = []
        			sparqler.execute(model, queryString) { QuerySolution soln ->
        				RDFNode sub = soln.get("sub");
        				RDFNode url = soln.get("url");
        				RDFNode type = soln.get("type");
        				RDFNode name = soln.get("title")
        				RDFNode df = soln.get("df")
        				def feed = new NewsFeed(name: name.asLiteral(), url:url.asLiteral(),
        					type:type.asLiteral(), dateFormat: df.asLiteral())
        				feeds += feed
        			}
        			exchange.out.body = feeds
        		}
        		.choice().when{Exchange e -> e.in.body.type == "rss"}
        					.to("direct:rss")
        				 .when{Exchange e -> e.in.body.type == "atom"}
        					.to("direct:atom")
        

It is not all plain sailing however. I would offer a couple of words of warning before diving in.

Firstly, the online documentation is, after months of reading, just about enough to work with. It is very tricky to get started with just the online document. The Component description pages are slightly different and never seem to give you all the information you need. The good news is that features and capabilities are generally supported, but just not well documented. So stick in. (Camel 3.0 will focus on documentation).

The other problem I have seen is that moving from a traditional coding it "all from scratch" to thinking in terms of messages, flows and integration points using a suite of pluggable components is not always an easy step for developers to make. It takes practice and plenty of trial an error. Normally if something is too difficult, there is a vastly better approach that you just may not be aware of yet.

A good starting point, to get people used to the principles and components, would be to try using Pentaho Kettle to build some ad-hoc flows. I used this for a year or two before starting on Camel, and it was definitely a good grounding.