Thursday, September 30, 2010

CXF Continuation API Enhancement

From Web 2.0, there are more and more long time connections are used by client to access the back end system, and they will consume lots of thread if the server is working on per thread per request mode. Jetty 6 introduced the Continuation API to resolve the this problem.

Few years ago CXF provides the same kind of continuation API to let the user have a chance to release the request thread if the response is not ready. This feature is very important if you want to keep your application with a good performance and good scalability.


As we did some house clean up from Camel 2.4.0 to make sure the Camel component has the best usage of the Camel Async Route Engine, I did some work on camel-cxf recently. I just found current CXF continuation which suspending is implemented by throw the runtime exception. This implementation has a shortcoming which cannot call the other framework's async API after continuation suspend is called as Jetty7 does. This will introduce a situation that continuation resume will be called before the continuation suspend is called.
To avoid this bad situation which will make suspended continuation never resumed, I had to write my Camel code like this

private Object asyncInvoke(Exchange cxfExchange, final Continuation continuation) {
synchronized (continuation) {
if (continuation.isNew()) {
final org.apache.camel.Exchange camelExchange = perpareCamelExchange(cxfExchange);

// use the asynchronous API to process the exchange
boolean sync = getAsyncProcessor().process(camelExchange, new AsyncCallback() {
public void done(boolean doneSync) {
// make sure the continuation resume will not be called before the suspend method in other thread
synchronized (continuation) {
if (LOG.isTraceEnabled()) {
LOG.trace("Resuming continuation of exchangeId: "
+ camelExchange.getExchangeId());
}
// resume processing after both, sync and async callbacks
continuation.setObject(camelExchange);
continuation.resume();
}
}
});
// just need to avoid the continuation.resume is called
// before the continuation.suspend is called
if (continuation.getObject() != camelExchange && !sync) {
// Now we don't set up the timeout value
if (LOG.isTraceEnabled()) {
LOG.trace("Suspending continuation of exchangeId: "
+ camelExchange.getExchangeId());
}
// The continuation could be called before the suspend
// is called
continuation.suspend(0);
} else {
// just set the response back, as the invoking thread is
// not changed
if (LOG.isTraceEnabled()) {
LOG.trace("Processed the Exchange : " + camelExchange.getExchangeId());
}
setResponseBack(cxfExchange, camelExchange);
}

}
if (continuation.isResumed()) {
org.apache.camel.Exchange camelExchange = (org.apache.camel.Exchange)continuation
.getObject();
setResponseBack(cxfExchange, camelExchange);

}
}
return null;
}



As you know CXF has lot of interceptor which can chained together as a SOAP stack and doing the WS* work, if we can suspend the interceptor chain, it will be easy for us to resume it from other thread. You can find a handy example from OneWayProcessorInterceptor.
My enhancement on CXF continuation API by introducing a suspend status into CXF PhaseInterceptorChain. The implementation of the CXF continuation API set the status of PhaseInterceptorChain, then the interceptor calling thread will be broke out after the interceptor handle method returns. Now my Camel code would be more easy to understand

private Object asyncInvoke(Exchange cxfExchange, final Continuation continuation) {
if (continuation.isNew()) {
final org.apache.camel.Exchange camelExchange = perpareCamelExchange(cxfExchange);
// Now we don't set up the timeout value
if (LOG.isTraceEnabled()) {
LOG.trace("Suspending continuation of exchangeId: "
+ camelExchange.getExchangeId());
}
// now we could call the suspend here
continuation.suspend(0);

// use the asynchronous API to process the exchange
getAsyncProcessor().process(camelExchange, new AsyncCallback() {
public void done(boolean doneSync) {
if (LOG.isTraceEnabled()) {
LOG.trace("Resuming continuation of exchangeId: "
+ camelExchange.getExchangeId());
}
// resume processing after both, sync and async callbacks
continuation.setObject(camelExchange);
continuation.resume();
}
});

}
if (continuation.isResumed()) {
org.apache.camel.Exchange camelExchange = (org.apache.camel.Exchange)continuation
.getObject();
setResponseBack(cxfExchange, camelExchange);

}
return null;
}

The enhanced CXF continuation API patch is committed into CXF trunk and it is part of Fuse Service FrameWork 2.3.0 now.

Monday, February 15, 2010

Configure the camel-cxf endpoint advance features from URI

As you know CXF provides the advance configuration on the CXF endpoints, such as interceptors, features etc.

In camel-cxf we leverage the CXF configuration which is based on the Spring to provide the same thing as CXF does, but in Camel we have the classical URI configuration which means you can configure the endpoint's by using the URI parameters. URI is easy way for the user as it support the DSL and Spring configuration smoothly, and you can deal with the URI as string to add your customer properties.

How can we configure the interceptor or the features on the camel-cxf endpoints by just setting few option parameters?
The key of the answer is you can set the bus of the camel-cxf endpoint by using the URI option bus.
You can set the features or interceptor on the bus as CXF bus configuration does and then set the camel-cxf endpoint URI like this "cxf://Address?Bus=#cxf&...".

Sunday, May 3, 2009

Better OSGi integration test support In Camel

Two weeks ago, James did a great work by adding an OSGi integration test module which was based on Pax Exam.

Then, I did some enhancements for this test module this week.

1. You can use the camel features to install the bundles for you test.
Instead of adding the camel-xxx module dependency in the POM for the Pax Exam plugin to setup the bundles list, we can leverage the camel features to setup the bundles that we may use in the integration test.

Here is a code example

// using the features to install the camel components
scanFeatures(mavenBundle().groupId("org.apache.camel.karaf"). artifactId("features").versionAsInProject().type("xml/features"),
"camel-core", "camel-osgi", "camel-spring", "camel-test"),


You can keep on adding the features that you need such as "camel-jms", "camel-jetty", and don't need to any modification on the module's pom.xml.

2. I added the OSGi version of the Camel TestSupport classes, so you could write the test more easily. If you are using the traditional Java DSL to setup the route rule, you just extend the OSGiIntegrationTestSupport; if you are big fan of Spring, you can extend the OSGIIntegrationSpringTestSupport, and it will take care of creating the application context in the OSGi plateform.

Thursday, March 12, 2009

Set the properties for Camel Context

If you want to do customer configuration on the inner component of Camel 2.0, how can you achieve that? You may say to use the system properties is a easy way.
But if I want to configure the component per camel context, as you know there could be more than one camel context per JVM.
I just introduced a property configuration per camel context in the Camel 2.0, so you want to configure the cachedOutputStream of Camel , you can do it like this.

With the java code

Map<String, String> properties = new HashMap<String, String>();
properties.put(CachedOutputStream.THRESHOLD, "1000");
properties.put(CachedOutputStream.TEMP_DIR, "/tmp/camel");
camelContext..setProperties(properties);



With Spring configuration

<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<properties>
<property key="CamelCachedOutputStreamThreshold" value="10000"/>
</properties>
<property key="CamelCachedOutputStreamOutputDirectory" value="/tmp/camel"/>
</properties>
<property key="the key of properties" value="the value as string"/>
</properties>
...
<camelContext>

Monday, February 23, 2009

How to use extra camel componets in servicemix-camel

When I was doing the spring clean up work for fuse ESB 3.x. I came across the issues of using the extra camel components within ESB3.x's servicemix-camel component.

Since Servicemix3's has same hierarchies of the class loaders with the J2EE application server, and the components the class loader is separated with the SU's. And Camel has lots of components, we just include the camel-core and camel-spring components in the servicemix-camel by default. When the user want to use other camel component, they always add the camel-xxx component into their SU lib.
In this way if there is a class which is loaded from the different SU classloader and registered into the camel-core's registry by servicemix-camel component , we will faced on the typical class cast exception in Servicemix3. Basically this exception is caused by the same class is loaded by different class loader.

Let me give you a real world issue.

In this case, when the SU redeployed, it will create a new deployer which will create a application context with the class loader of servicemix-camel component, and then using the SU's class loader to create a camel context.
Since the SU is redeployed, Servicemix will create a new class loader to load the SU's lib jars and resources.
When the SU initial the rmi component, boom , we meet the situation of same component loaded with different class loader, The class cast exception is thrown out.

Note, In Servicemix4 , we will not get that kind of issue , since the OSGI will help us to manage the relationship of the jars :).

The solution is putting the camel-xxx component and third part jars into servicemix-camel component's lib. You need to check out the servicemix-camel component's pom.xml and recompile the servicemix-camel component by adding the camel-xxx component dependency.

Let me take the latest servicemix-camel as an example

1. Checking out the servicemix-camel component's pom


svn co http://svn.apache.org/repos/asf/servicemix/components/engines/servicemix-camel/trunk/pom.xml


2. If you need camel-rmi component , you just put the dependency into the pom.xml


<dependency>
<groupid>org.apache.camel</groupid>
<artifactid>camel-rmi</artifactid>
<version>${camel-version}</version>
</dependency>


3. Running "mvn install" and copy the servicemix-camel-*.zip into the deploy directory

4. Write your servicemix-camel SU, make sure you don't include any camel relates jars into the SU's lib, you can use the <scope>provided</scope> to not packing the artifact.


<dependency>
<groupid>org.apache.camel</groupid>
<artifactid>camel-rmi</artifactid>
<version>${camel-version}</version>
<scope>provided</scope>
</dependency>

Thursday, January 22, 2009

Powerful svn merge

Since I'm always using svnmerge.py to do the merge stuff, I have no chance to go through the svn merge doc.
Now I just found svn merge is so powerful.

If you just want to merge one pic of change from trunk to branch, you don't have to do the
svnmerge.py init

svnmerge.py merge ...

You can just type in you working copy directory

svn merge -c VERSION YOUR_TRUNK_URL [WCPATH]


If you want to revert the change for the svn server side , not just your working copy
svn merge -rVERSION:VERSION-1 SOURCE [WCPATH]

Wednesday, January 21, 2009

What's new in Camel 2.0

I have been asked by lots of friend who are using or going to using Camel, what's new in Camel 2.0.
I just found Claus set up a good wiki page for all these kind informations.
Some high lights includes:

New features:
  • Make more endpoints easily configurable as beans in Spring XML.
  • Improve the DSL
  • ...
API Changes
  • Remove the use of generics on Component, Exchange, Producer, Consumer
  • Change to use verb for EIP action
  • ...


You can find the release notes about Camel 2.0 (Working in progress) here.