Concurrent Programming — Introduction

What is Concurrency?

Concurrency is the ability to run several programs or several parts of a program in parallel. If a time consuming task can be performed asynchronously or in parallel, this upgrades the performance of the program by increasing the throughput and the interactivity of the program.

Few years back, computers didn’t have operating systems, they executed a
single program from start to end, and that program had direct access to all the resources of the machine. Executing a single program at a time was an inefficient use of expensive and scarce computer resources.

A modern computer has several CPU’s or several cores within one CPU. The ability to leverage these multi-cores can be the key for a successful high-volume application.

Several factors led to the development of operating systems that
allowed multiple programs to execute simultaneously:

• Resource utilization — Sometimes programs have to wait for external operations. It is more efficient to use that wait time to let another program run.
• Fairness — Multiple users and programs may have equal claims on the machine’s resources. It is preferable to let them share the computer via finer-grained time slicing than to let one program run to completion and then start another.
• Convenience — It is often easier or more desirable to write several programs that each perform a single task and have them coordinate with each other as necessary than to write a single program that performs all the tasks.

The same concerns (resource utilization, fairness, and convenience) that motivated the development of processes also motivated the development of threads.

What is Thread?

A Thread is a

• Facility to allow multiple activities within a single process.
• A thread is a series of executed statements.
• A thread is a nested sequence of method calls.
• Referred as lightweight process.
• Each thread has its own program counter, stack and local variables.

Why we use Threads?

• Threads help to perform background or asynchronous processing.
• It increases the responsiveness of GUI applications.
• Threads take advantage of multiprocessor systems.
• It simplifies program logic when there are multiple independent entities.

Threads in Java

Every Java program creates at least one thread [ main() thread ]. Additional to that , threads can be created through the Thread constructor or by instantiating classes that extend the Thread class.

In java thread can be created in two ways.

• Extending the java.lang.Thread class.
• Implementing the java.lang.Runnable Interface.

Benefits of Threads

1. Utilizing multiple processors

Few years back, multi processor systems are considered as a rare and expensive in the industry.

It’s only used large data centers and scientific computing facilities. But now, they are cheap and plentiful, even low-end server and mid range desktop systems often have multiple processors.

Since the basic unit of scheduling is the thread, a program with only one
thread can run on at most one processor at a time. On a two-processor system, a single-threaded program is giving up access to half the available CPU.
resources.

On the other hand, programs with multiple active threads can execute simultaneously on multiple processors. When properly designed, multi-threaded programs can improve throughput by using available processor resources more effectively.

Using multiple threads can also help achieve better throughput on single-processor systems.

Lets say, If a program is single-threaded, the processor remains idle while it waits for a synchronous I/O operation to complete. In a multi-threaded program, another thread can still run while the first thread is waiting for the I/O to complete, allowing the application to still make progress during the blocking I/O.

In real time scenario, this is like reading the newspaper while waiting for the water to boil, rather than waiting for the water to boil before starting to read :)

2. Simplicity of modeling

Lets start with the real time example. One day your project manager has assigned you only one type of task to perform. Lets say he asked you to fix ten bug fixes. On the other day he has assigned you multiple type of tasks. Lets say he asked you to fix three bug fixes, asked you to conduct an interview for the interns, asked you to complete the performance evaluation of your subordinates and asked you to prepare for the client presentation.

Now think, when you have only one type of task (ten bug fixes) s to do, you can start from the fist bug fix, and keep working till the last one. you don’t have to spend any mental energy figuring out what to work on next.

On the other hand, managing multiple priorities and deadlines and switching between from one to other one carries some overhead and will affect your performance.

The same is true for software as well, a program that processes one type of task sequentially is simpler to write, of course less error-prone, and easier to test than one managing multiple different types of tasks at once.

Assigning a thread to each type of task or to each element in a simulation affords the illusion of sequential and insulates domain logic from the details of scheduling, interleaved operations, asynchronous I/O, and resource waits.

A complicated, asynchronous workflow can be decomposed into a number of simpler, synchronous workflows each running in a separate thread, interacting only with each other at specific synchronization points.

3. Simplified handling of asynchronous events

A server application that accepts socket connections from multiple remote clients may be easier to develop when each connection is allocated its own thread and allowed to use synchronous I/O.

If an application goes to read from a socket when no data is available, read
blocks until some data is available. In a single-threaded application, this means that not only does processing the corresponding request stall, but processing of all requests stalls while the single thread is blocked.

To avoid this problem, single-threaded server applications are forced to use non blocking I/O, which is far more complicated and error-prone than synchronous I/O. However, if each request has its own thread, then blocking does not affect the processing of other requests.

Risk of Threads

Thread safety / correctness
The main source of thread safety problem is the shared variable/resource. The solutions:

• don’t share variable (e.g. stateless servlet)
• make variable immutable
• use lock (synchronize in Java)

Race condition is the most common concurrency correctness problem. Pay attention for the compound actions (e.g. check then act, read then write). Using thread safe atomic objects don’t always guarantee the correctness, the compound actions themselves need to be atomic (e.g. using synchronize).

Deadling with performance problems due to locking

Improper / excessive lock usages can lead to liveness hazard & performance hazard. Tips to work with locking:

• reduce the lock scope, for example: synchronize a block of codes instead of the whole method, delegated lock (e.g. using thread safe objects)
• avoid locking lengthy operations (waiting user response, contended resource, long distance web service)
• don’t nested lock
• Lock splitting / reduce lock granularity: use different locks for each object instead of using one lock in a long code.
• use alternatives to locking: confinement / don’t share object (e.g. use local variables, deep copy to create local copy for each thread, use Thread Local), immutable object (final), volatile object.
• provide deadlock resolution (e.g. timeout and then choose a victim)

Deadlock causes

• Inconsistent lock ordering (e.g. thread 1 acquires lock on resource A then lock on resource B while thread 2 acquires lock B then A, so thread 2 will wait for thread 1 and vice versa) . Solution: provide global ordering (e.g. bash on hash code of the objects) instead of using dynamic/input-dependent order.
• Calling an locking alien method within a lock (double locking). Solution: open call (call without lock)
• The limitation of resource capacity (e.g. we have 100 requests while the resource pool limit the connection to 10 so the other 90 have to wait). Solution: bigger capacity resource, clustering & load balancer, request throttling.
• The thread is waiting for another thread result (that might a lengthy operation or even never finish). Solution: time out, refactor the flows to avoid waiting if possible, use Future task so that the computation can be started even before it’s needed.