Why do we scale more if we use less threads?

Threads are costly resources and are limited in the sense of how many you can create and still have a responsive system. More threads represent more context switching at the CPU and more work executed by the system, leaving fewer resources to execute what matters: application code. Threads are also expensive to be created, that’s why we generally see pools of them, so releasing a thread quicker will enable you to be able to handle more requests and will, certainly, improve the scalability of your entire system.

What does OSB provide?

When routing to a service which has a non blocking capable transport, depending on the scenario (protocol, type of routing, etc), OSB has the capability to send the request and release the thread until a response is received. This is extremely useful in cases where the response may be too slow so that the released thread may be used to serve other request instead of keeping blocked for a long time. When the callee service has a response to deliver, OSB borrows a thread from WebLogic’s thread pool and uses it to handle the response. OSB has support for a lot of protocols as non blocking and we will explore how this works with HTTP.

Seeing it in practice

Creating a slow service

To see what happens, we first need a slow service, which will allow us to route to it and see the request thread being released back to the pool. This is achieved by the following JAX-WS service which sleeps for 60 Seconds before delivering the response:

package br.com.amadei.osb.nbhttp;
import java.util.Date;
import javax.jws.WebService;

@WebService
public class SlowWS {

	public String verySlowOperation() {
		try {
			Thread.sleep(60000);
		} catch (InterruptedException e) {
			e.printStackTrace();
		}
		return new Date().toString();
	}
}

Deploy the WAR containing this Web Service to the same WLS running OSB.

We will have another class, which will be called via a Java Callout from OSB to show the current thread. The class is as follows:

package br.com.amadei.osb.nbhttp;

public class LogThread {

	public static void logCurrThread() {
		System.out.println("***************************** "
				+ Thread.currentThread());
	}
}

Compile this class and create a JAR. In OSB, create an OSB project and copy this JAR into it. It will be used later.

Now, import the WSDL and XSD of the SlowWS to OSB and create a business service for it as can be seen bellow. This OSB business service should point to the created "SlowWS" JAX-WS web service, deployed as a WAR.

Create a proxy service based on the same WSDL and add a Routing to the created business service. Inside the request and response actions of the Routing, add a java callout to the LogThread.logCurrThread method call. This can be seen bellow:

Create a soapUI project to test the proxy service as testing it via SBCONSOLE won’t release the thread as it will remain active for the web console, so soapUI is a better option in this case.

Before making the first request, go to the WebLogic console. We will take a thread dump to see the state of the thread which served the request. So, after logging in, expand the Environment link and click Servers. Select the AdminServer (supposing you’re running the developer install of OSB), Monitoring, Threads. This is where you’ll be able to take the dump when needed.

Before going back to soapUI to make the request, notice that it will log the thread handling the request, so you will be able to look back at the dump. You should do the following:

1. Invoke the request via soapUI;
2. Go back to the console and take the name of the thread OSB logged to the system output;
3. Take a thread dump and see that this thread is doing;

Let’s try it!

After starting the request via soapUI, I see the following message in the standard output:

***************************** Thread[[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)',5,Pooled Threads]

This means that the ExecuteThread 5 from weblogic.kernel.Default is handling my request. After that, I took a thread dump and searched for this thread and could see the following stack for it:

[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock weblogic.work.ExecuteThread@42f9ef WAITING
    java.lang.Object.wait(Native Method)
    java.lang.Object.wait(Object.java:485)
    weblogic.work.ExecuteThread.waitForRequest(ExecuteThread.java:205)
    weblogic.work.ExecuteThread.run(ExecuteThread.java:226)

This stack means this thread is waiting for a request inside WebLogic’s Thread pool or, to be more specific, it was returned to the pool. At this time, the response was not yet to our service and the client is also waiting OSB to deliver the response but, the important thing to notice is that we are not holding a thread until such response available, proving the non blocking capability.

After 1 minute, the response is delivered and we see in the log the thread elected to hold the response (even if the request was blocking, response would have been delivered in a different thread).

The other side

This behavior may seem a little strange at the first seen, so we can see how the behavior would be if this request was blocking. To make it blocking, add a Routing Options node inside the Request Action pipeline and configure its QoS as “Exactly Once” as can be seen in the figure bellow:

The Routing will be blocking if the QoS is exactly once (Service Callout calls are also always blocking).

Deploy the new version and do the same steps done before. This time, Execute Thread 14 was elected to serve my request, and looking at its stack, we see it’s blocked trying to read the response.

[ACTIVE] ExecuteThread: '14' for queue: 'weblogic.kernel.Default (self-tuning)'" RUNNABLE native
   java.net.SocketInputStream.socketRead0(Native Method)
   java.net.SocketInputStream.read(SocketInputStream.java:129)
   java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
   java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
   java.io.BufferedInputStream.read(BufferedInputStream.java:317)
   weblogic.net.http.MessageHeader.isHTTP(MessageHeader.java:224)
   weblogic.net.http.MessageHeader.parseHeader(MessageHeader.java:148)
   weblogic.net.http.HttpClient.parseHTTP(HttpClient.java:468)
   weblogic.net.http.HttpURLConnection.getInputStream(HttpURLConnection.java:401)
   weblogic.net.http.SOAPHttpURLConnection.getInputStream(SOAPHttpURLConnection.java:37)
   weblogic.net.http.HttpURLConnection.getResponseCode(HttpURLConnection.java:1005)
   com.bea.wli.sb.transports.http.HttpOutboundMessageContext.getResponse(HttpOutboundMessageContext.java:679)
   com.bea.wli.sb.transports.http.HttpOutboundMessageContext.send(HttpOutboundMessageContext.java:347)
   com.bea.wli.sb.transports.http.wls.HttpTransportProvider.sendMessageAsync(HttpTransportProvider.java:211)

Here, we saw how the thread could get blocked until a response is received. As we saw during the post, this is worst than the former scenario, so try to leverage non blocking whenever possible - sometimes, however, you have only the option of working with blocking requests.

Conclusion

Oracle Service Bus offers a extremely helpful way for you to scale your system and what’s better is that it requires no coding. Take this into consideration when architecting your services in OSB!

Suppose you have a service in OSB with the following pipeline:

This service will always raise an error and, as this is an WSDL based HTTP web service, this will translate into a SOAP Fault being returned to the called. The contents of the SOAP Fault will be dictated by OSB with some customization made in the Raise Error stage action as can be seen bellow:

When calling this service (let’s name it CalleeService), we get the following response (as expected):

Now, let’s try to route to the Callee Service from this new one. Let’s call it Caller Service and its routing configuration should be like the one bellow:

With the routing configured, let’s try to call our new created Caller Service. You must be expecting to receive the same error thrown by Callee Service in respone, however, that’s not what happens as can be seen bellow:

OSB handles the error and returns a generic protocol error.

To make OSB propagate the error received, add an Error Handler to the flow. We will add it at the service level as can be seen bellow:

Add a Reply inside the Error Handler and configure it as “With Failure” as can be seen bellow:

Now, executing the service, we see the error is propagated up to the caller as thrown by the callee service, what is the main objective of this post. Bellow, we see it:

I aim to have successfully shown you how to propagate errors in OSB in a very simple manner. Obviously you can write code to get more advanced behavior but if you need to simply propagate the error and avoid the generic error handling made by OSB, this post will certainly help you!

You can do such registration via the servlet context and there’s also a way to directly detect servlets from a JAR file.

Bellow we have a sample servlet that we will register dynamically with the web container:

package br.com.amadei.blog.dyn.servlets;

import java.io.IOException;
import java.util.Date;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class ServletA extends HttpServlet {
	private static final long serialVersionUID = 1L;

	protected void doGet(HttpServletRequest request,
			HttpServletResponse response) throws ServletException, IOException {

		response.getWriter().println("<h1>");
		response.getWriter().println("Hi, it's " + new Date());
		response.getWriter().println("</h1>");
	}

}

We can register such servlet via a listener like the one bellow:

package br.com.amadei.blog.dyn.servlets;

import javax.servlet.ServletContext;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.ServletRegistration.Dynamic;
import javax.servlet.annotation.WebListener;

@WebListener
public class MyContextListener implements ServletContextListener {

	@Override
	public void contextDestroyed(ServletContextEvent event) {
	}

	@Override
	public void contextInitialized(ServletContextEvent event) {
		ServletContext context = event.getServletContext();

		Dynamic dynamic = context.addServlet("ServletA", ServletA.class);
		dynamic.addMapping("/ServletA");
	}

}

There are some interesting things to notice:

• We use the new annotation made available by the Servlet 3.0 specification, @WebListener so that our listener is discovered without the necessity of being declared in the deployment descriptor (we can also add listeners dynamically via method calls too!).
• The method responsible for the servlet registration is ServletContext.addServlet(). It returns an instance of javax.servlet.ServletRegistration.Dynamic that is used to configure the servlet registration later (in this case, we use it to define the servlet’s URL mapping).

By running the application and pointing the browser to <server address>:<server port>/<Context Root>/ServletA, we see the programatically registered servlet responding, showing our sample just worked as shown bellow:

There are many other programatic changes you can make new to servlet 3.0:

• Register and configure servlets programatically (as we saw);
• Register and configure listeners and filter programatically, in the same way we did with servlets;
• Configure the session tracking cookie;
• Configure the session tracking mode.

This sample may seem useless and in fact it really is. My main idea was just to show this capability and I believe you can imagine how powerful it can be for framework and product builders that rely on the Servlet API.

Access it clicking here.

Let’s see it in practice.

Imagine you have the following XSD defining your input (process element) and output (processResponse).

And you have a BPEL service capable of returning the rest of the person’s info based on the informed ID. Your BPEL process should look like the following:

In the enrich assign we assign hard coded values to the name and age elements (it’s just a test...). We enrich the received person and copy it to the output as shown bellow:

The following SOAP request is made to the BPEL process:

You should be willing to receive a person as response containing name and age, however, you should receive an error similar to the one bellow:

faultName: {{http://schemas.xmlsoap.org/ws/2003/03/business-process/}selectionFailure} messageType: {{http://schemas.oracle.com/bpel/extension}RuntimeFaultMessage} parts: {{ summary=<summary>XPath query string returns zero node. The assign activity of the to node query is returning zero node. Either the to node data or the xpath query in the to node was invalid. According to BPEL4WS spec 1.1 section 14.3, verify the to node value at line number 76 in the BPEL source. </summary>}


The reason for such error is that the xpath /client:process/client:person/client:name did not evaluate to a single node but to zero.

One option would be to create the node empty with a “XML fragment” before assigning to it but it can become quite tedious. A better option is to use the Oracle extension property bpelx:insertMissingToData. To use such property, right click the copy line and select the insertMissingToData from the menu as shown bellow:

Do the same for the age assign and click Apply. Your assign should look like the one bellow:

Now, the BPEL process run should be successful, showing the bpelx:insertMissingToData helped a long way!

This was made easy by the possibility of running Groovy scripts as a test step and can help you to configure a parameter file or a properties file before running some piece of test.

So, to see this in practice, create a new soapUI Workspace and a new empty Project inside it. Add a new Test Suite to the project:

And add a new Test Case to the Test Suite:

Add a new Groovy Script step to the test case:

Inside the Groovy Script Test Step, enter the following source code:


File file = new File("/tmp/myConfig.properties")
file.delete()
file << ("PROPERTY1=123")
file << ("\r\n")
file << ("PROPERTY2=456")

This piece of code will:

• First open file at “/tmp” called myConfig.properties
• Delete it (if it exists)
• Then we write 2 key/value properties to the file, breaking the line between both of them.

The source code should look like the following image:

Now, run the script and you should see an information alert showing it was successful. Go to the file system and open the /tmp/myConfig.properties file. You should see it was generated as expected like shown in the image bellow:

As you may have noticed, this was pretty easy and this possibility enables you to change properties so that your tests can avoid human intervention to change property and other kinds of files.

The code bellow shows how to force only 5 threads to run concurrently even when we have 10 threads running.

package semaphore.tests;

import java.util.Date;
import java.util.concurrent.Semaphore;

public class SemaphoreTests {

  public static void main(String[] args) {
    Semaphore semaphore = new Semaphore(5);

    for (int i = 0; i < 10; i++) {
      new Worker(semaphore).start();
    }
  }
}

class Worker extends Thread {
  private Semaphore semaphore;

  public Worker(Semaphore semaphore) {
    this.semaphore = semaphore;
  }

  @Override
  public void run() {

    while (true) {
      try {
        semaphore.acquire();

        String msg = Thread.currentThread().getName() + " - running at: "
            + new Date();

        System.out.println(msg);

        Thread.sleep(10000);

      } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
        break;
      } finally {
        semaphore.release();
      }
    }
  }
}

When running such code we see the following:

Thread-3 - running at: Thu Oct 20 23:38:48 GMT-03:00 2011
Thread-1 - running at: Thu Oct 20 23:38:48 GMT-03:00 2011
Thread-0 - running at: Thu Oct 20 23:38:48 GMT-03:00 2011
Thread-5 - running at: Thu Oct 20 23:38:48 GMT-03:00 2011
Thread-4 - running at: Thu Oct 20 23:38:48 GMT-03:00 2011

Thread-1 - running at: Thu Oct 20 23:38:58 GMT-03:00 2011
Thread-5 - running at: Thu Oct 20 23:38:58 GMT-03:00 2011
Thread-0 - running at: Thu Oct 20 23:38:58 GMT-03:00 2011
Thread-4 - running at: Thu Oct 20 23:38:58 GMT-03:00 2011
Thread-7 - running at: Thu Oct 20 23:38:58 GMT-03:00 2011

Thread-2 - running at: Thu Oct 20 23:39:08 GMT-03:00 2011
Thread-6 - running at: Thu Oct 20 23:39:08 GMT-03:00 2011
Thread-4 - running at: Thu Oct 20 23:39:08 GMT-03:00 2011
Thread-0 - running at: Thu Oct 20 23:39:08 GMT-03:00 2011
Thread-7 - running at: Thu Oct 20 23:39:08 GMT-03:00 2011

Thread-6 - running at: Thu Oct 20 23:39:18 GMT-03:00 2011
Thread-2 - running at: Thu Oct 20 23:39:18 GMT-03:00 2011
Thread-4 - running at: Thu Oct 20 23:39:18 GMT-03:00 2011
Thread-9 - running at: Thu Oct 20 23:39:18 GMT-03:00 2011
Thread-7 - running at: Thu Oct 20 23:39:18 GMT-03:00 2011

Notice that five threads run concurrently, sleep for five seconds and then another five threads run concurrently again and sleep for five seconds more and so on...

If you change the number of the Semaphore's permits to, for example, two permits, you should see only two threads running concurrently, sleeping for five seconds, then two more running and so on, proving that the Semaphore is controlling the throttling.

To change the number of permits, simply change the number that the Semaphore receives in its constructor: Semaphore semaphore = new Semaphore(2); and you should see:

Thread-2 - running at: Thu Oct 20 23:54:17 GMT-03:00 2011
Thread-3 - running at: Thu Oct 20 23:54:17 GMT-03:00 2011

Thread-2 - running at: Thu Oct 20 23:54:27 GMT-03:00 2011
Thread-0 - running at: Thu Oct 20 23:54:27 GMT-03:00 2011

Thread-2 - running at: Thu Oct 20 23:54:37 GMT-03:00 2011
Thread-4 - running at: Thu Oct 20 23:54:37 GMT-03:00 2011

Thread-2 - running at: Thu Oct 20 23:54:47 GMT-03:00 2011
Thread-7 - running at: Thu Oct 20 23:54:47 GMT-03:00 2011

Thread-2 - running at: Thu Oct 20 23:54:57 GMT-03:00 2011
Thread-7 - running at: Thu Oct 20 23:54:57 GMT-03:00 2011

We’ve seen how to implement a throttling mechanism using the Semaphore Java class. On the next post I’ll show how the Semaphore class can be used to control access to a pool of resources.