Subsections

14. Thread Based Models for Distributed Systems

JAVA WAS ORIGINALLY CONCEIVED BY SUN MIRCOSYSTEMS to be used on embedded systems. However, it turned out to be a better language for the Internet because it offered a virtual machine that could run Java applications on all kinds of different architectures. Java is an object oriented programming language that supports automated memory management (= garbage collection), threads, and interfaces. Libraries like Java RMI (remote method invocation) are available to support semi transparent remote accessibility. This model, a typical thread based, transparent distribution model, is the focus of this appendix. We will investigate the suitability of Java and its thread based model for writing adaptors. This chapter explains why we opted for an event based model. It gives also isnight into the problems of creating adaptors for such models.

14.1 Naming/Finding services

THE FIRST THING EVERY distributed application has to do before connecting a remote object, is looking up what object it should connect to. For this purpose Java RMI uses a special server which should be started on the machine where remote objects will be exported. This server is called the RMI registry.

If an object wants to export itself it can use the Naming class to export its own name to the registry. From then on all applications can ask the registry a reference for the object with that name.

It is clear that such a mechanism is very rudimentary, because we still need to know the name of the machine where the registry (and the objects as such) are running. To support new technologies, such as wireless embedded devices, Sun developed JINI, which allows peers (clients) to look up other peers (servers) using a description of the required capabilities, instead of a simple name. The initial lookup to find a JINI directory server is done with a broadcast.

14.2 Communication

**Figure 7:** Remote object calling in Java RMI.
$\includegraphics[% width=0.80\textwidth]{StubSkeletonFlow.eps}$

JAVA PROCESSES can communicate by using Java RMI. Note that Java RMI can only communicate between Java Processes, which is a major drawback of RMI.

Sun's serialization and deserialization interface to Java [CFKL91] helps with exporting an object graph to a byte stream. The standard behavior for serialization is to serialize the object and all the objects it contains. If we want to modify this behavior we have to implement an externalizable interface which describes how the object is to be exported.

To be able to contact a remote object, as if it were a local object, one should create stubs for all the remote objects which will be contacted.^14.1 Such a stub will implement all the methods the remote object supports and filling in the bodies with code which contacts the remote object. The logic of such a stub body is quite simple:

Contact the remote object using sockets
Serialize all the arguments passed to the stub by the caller.
Send out all arguments
Wait for an answer, which will be the result of the remote method call.
Receive/read the result
Deserialize the result
Return the result to the original caller

The object that is being exported at the server side, offers a way to be contacted by remote clients by means of a skeleton. This skeleton contains some listening code and logic to contact the actual exported object. The logic in a skeleton is as follows:

Listen for connections
Accept an incoming connection
Receive the serialized arguments
Deserialize the arguments
Invoke the method call upon the correct object
Serialize the result of the invocation
Send back the result over the socket

Figure 7 illustrates how we can transparently contact a remote object. We see how the setup is inherently client-server. When we want to contact a client from within the server the client needs to export an interface and become a listening server itself.

14.3 Openness & Remote Objects

ANOTHER POINT TO MAKE with respect to stubs and skeletons is that they are generated by a compiler, called rmic. This means that stubs and skeletons are created at compile time. As a direct consequence we cannot easily communicate with an unknown process at runtime. In the setting of this dissertation, this is not acceptable because we don't know the interface we will link to.

One could think of a solution by creating stubs and skeletons as needed: any time a connection to another machine is needed, the interface description could be downloaded from the server and compiled into a stub class that would connect to the appropriate skeleton at the server. This compilation phase would require a compiler or Java byte code assembler, and would consume a lot of time for simply setting up a link to a remote object. It is clear that this is not practical and not a good approach at all.

Another approach, as used by Smalltalk [AGR83] users, is using the meta-level interface and overriding a method such as doesNotUnderstand. With this a simple and general stub could be created. The only method of the stub would be the doesNotUnderstand. This method would be called every time an undeclared method is invoked upon the object as can be seen in figure 8. The doesNotUnderstand in turn would look at the actual method invocation and pass it along to the skeleton. However, as it turns out, this is impossible with Java because the meta-level interface of Java is not strong enough.

**Figure:** The behavior of a general stub, using `doesNotUnderstand`.

14.4 Java Threads

IN MANY ASPECTS JAVA is an innovating language. One of these innovations is the introduction of a standard threading library.^14.2 A thread is an execution context which can run together with other threads in the same environment. The difference with processes is that threads do share memory, while processes don't share memory.

$\begin{algorithm} % latex2html id marker 7699 [!htp] \begin{list}{}{ \setlengt... ...ar }\end{list}\par \caption{ A distributed recursive factorial.} \end{algorithm}$

The availability of threads in Java is of crucial importance for the internal workings of Java RMI as we will explain now. Java RMI, as already seen, waits before returning an answer: a client can ask the server to execute a method and return the answer. In the meantime the client simply waits. Now, let's have a look at the Java program in algorithm 27 (page ).

The FacApp class exports a FacCalcer interface. A FacCalcer is an object that calculates the factorial of a certain number by performing a recursive call. To do the recursive call the FacCalcer needs to receive another FacCalcer as can be seen in the definition of the fac method. When such a FacCalcer starts, it either becomes the master FacCalcer (by binding itself in the registry) or a slave FacCalcer, which will initiate the calculation of a factorial.

To start this program, first an rmiregistry should be running and afterwards two instances of the FacApp should be started. The last facapp (faccalcer2 from now on) requests the first faccalcer1 to calculate the factorial of 3. In return faccalcer1 will request faccalcer2 to calculate the factorial of 2... But how is this possible ? How can the original requester be interrupted while he is still waiting ? The answer lies in the Java threading mechanism. Every time an RMI call comes in, the server-thread will spawn a new thread which handles the request. This can be seen in figure 9.

**Figure 9:** In the above figure, full lines indicate a running process, dotted lines indicate a waiting (listening) process and horizontal dashed lines indicate a 'wait for return'. In this figure we see how a new thread is spawned every time a call comes in.

The result of this behavior is nicely what a programmer would expect. The problem is that this application suffers from a number of conceptual problems, which are essentially grounded in the inherent problems of distributed systems: concurrency and partial failure. We will explain this below.

14.5 Error handling: Exceptions

LET'S HAVE A LOOK at how failures of the underlying network and failures of processes are caught. Java has a well-known language construct of exceptions and uses this to report errors that occur when contacting a remote object. Technically, RMI achieves this by letting the stub throw an appropriate exception. When, on the other hand, the skeleton fails while executing the incoming message (because the program throws some kind of exception) it will simply serialize the exception and send it back to the client.

Although, this solution looks nice, there is not much that can be done when such an exception is caught. Do we reconnect with the server, do we inform the user, or what should we do ? The main problem programmers encounter here is how to handle those exceptions in a structured way.

To illustrate these problems, suppose we have a process that calls another process , which in its turn will contact process again. What will happen when process dies at the moment is sending back its result to ? Will process know in what state $T_{0}$ is ? How will the exceptions cascade ? Figure 10 illustrates this. At the moment $T_{1}$ , i.e. at the moment process received a broken pipe exception from the underlying socket layers, it does not know anymore that it originally received a call from process . Process cannot easily know that an internal thread is still executing or not. The net result is that process ends up to be in an unknown, probably invalid, state.

**Figure 10:** An illustration what can go wrong with the nested calling conventions of Java RMI. At the moment process dies, process does not know whether has already been executed or not. This leaves in an unknown state.

14.6 Java RMI and Concurrency

14.6.1 Concurrency Primitives

SINCE JAVA IS A LANGUAGE with native support for threads, we need to investigate how concurrency can be managed and what kind of language constructs are available. The first and most important language construction is synchronized.

  synchronized(foo)
    {
    int val=foo.read();
    foo.write(val+1);
    }

In Java all objects can have a lock, when an object is synchronized the current thread will try to obtain a lock on that object, and if the lock is obtained, the statement block will be executed. When leaving the block statement the lock is released again. The object locks are reentrant, so the same thread can lock the same object multiple times. With this construct one can easily implement a critical section. However, it is still allowed that other accesses to the foo object are not synchronized, thereby ignoring concurrency behavior.

A second construct is the possibility to synchronize methods in an object. For example:

  synchronized public void increase()
    {
    ...
    }

Which means that the increase method will only execute when the this object is locked. In fact we can write exactly the same as follows:

  public void increase()
    {
    synchronized(this)
      {
      ...
      }
    }

Now, although a nice construct it suffers the same problems as all concurrency primitives in object oriented languages: the inheritance anomaly. Suppose, we specify a method in a class to be synchronized, this means that all overriding methods must be synchronized too. This effectively means that a subclass cannot choose to be not synchronized for its own actions, and be synchronized for the super calls. Aside from this annoying problem, there are a lot of other problems with respect to synchronization and concurrent object oriented languages.

Note that, aside from these (relatively low level) synchronization mechanisms, there are other solutions like wait-notify mechanisms, the Java Transaction interface which offers a much more high level approach to concurrency strategies and others. For a more detailed discussion about concurrency management and Java see [Lea00].

14.6.2 Java RMI

THE JAVA THREADING MECHANISM and the way Java RMI uses it, makes it possible for one remote object to be invoked multiple times by different threads on a concurrent basis. This means that the object's state will be soon invalid if we don't guard access to the remote method.

If we now use the standard Java keyword synchronized to guard access to the remote object, we see that only one thread can enter the remote object at a time, thereby placing all other threads in wait until the object becomes available again. However, this still raises some problems.

**Figure 11:** In the above figure we see how a deadlock occurs between two synchronized RMI calls, which normally would work if not distributed.

For example, let us take the previous factorial example and assume that the fac method is synchronized. What will happen ? One would expect the program to produce its standard result 3,2,1,... and so on. In practice this will not happen because the thread that comes back from the first faccalcer is different from the thread that is waiting inside a synchronized block. So this new thread will have to wait for the lock to be released and finally deadlock because this new thread is supposed to offer an answer before the calling thread will release the lock. In figure 11 we see the control flow of this program.

This example illustrates that we still need some active form of session management in distributed systems. It also illustrates how concurrency problems cannot simply be solved by synchronizing methods. In distributed systems every remote interface will need to export some kind of concurrency interface, and the implementation will need to have a well thought off concurrency management strategy. This concurrency management is typically larger than the actual actions to perform. We will come back on these issues in detail in chapter 5.

14.7 Writing Adaptors

WE HAVE NOW SEEN how Java RMI works. We have seen how concurrency is managed and how threads are used. Writing adaptors with Java RMI is clearly not as easy as with the component system we have been using.

The Java RMI call-wait-return calling conventions makes implementing an adaptor difficult. An adaptor can receive multiple incoming calls and will spawn multiple internal threads in response. These threads need to be guarded somehow. Writing this guarding mechanism is not trivial since the threads themselves are implicitly started.
The Java RMI registry makes it difficult to plug in an adaptor between two communicating processes. When one process contacts another it lookup the name of the remote object. If we could rebind this name to point to the address of an adaptor this could be possible. However it is impossible to rebind a name (let us say Server) to a new location (Adaptor). This is necessary to make sure that all processes wanting to contact the original Server will contact our Adaptor.
A typical Java RMI connection only sends data and receives one answer. If we want to adapt data sent over connections between different processes, our adaptor will need to implement both directions. It will need to listen to the first process, as well as to the second process. All incoming connections must be adapted and identification of the correct remote object should be present. An important issue here is that we don't know beforehand which objects are being exported by a Java Process and whether they should be adapted or not.
At runtime we cannot contact a process unknown at compile time because the Java RMI reflection interface is not good enough to be able to make generic stubs.
The concurrency guarding implemented in a remote object is not visible to the outside world, nevertheless it has a strong impact in how the application works and will respond to incoming calls. The adaptor should work together with the synchronization behavior of client and server. But since the concurrency interface is not exported this cannot be achieved.

Footnotes

...^14.1: This can be automatically done using rmic.
...^14.2: compare this with C which has all kinds of dirty libraries and tricks like setjmp and longjmp to handle multiple sessions.

Werner 2004-03-07