The first part of this article presented a simple application for photo sharing. Let’s see how Gearman would be useful in this case.

A gearman system has three components:

• A client which creates jobs to be run and then, sends them to a job server. Gearman provides a client API that your applications can use to communicate with the job server.
• A worker which registers jobs. It can run to a job server and grab jobs to run it.
• A job server, called Gearmand, which coordinates the assignment from clients to workers and supervises the job.

Gearman system architecture

So let’s take a look at how the applications would look using a Gearman system.

This is a simple schema where the focus is on the Gearman part of the application. Database, mail, cache and servers are omitted and could be added to this application. We have the http servers, where the Gearman clients are located, and servers with worker managers and worker servers, which do the actual work. In our case the tasks are resizing images and sending emails. A client makes a task for each image and when all images are resized the email tasks are created.

Failure Cases

There are some failure scenarios:

When an http server goes down, the resize tasks are already in Gearman so they continue to be executed and the results can be retrieved by another http server. Gearman also lets clients recover a job if they lost connection with the job server. Each task has a unique id, if two jobs with the same id are given. The task is executed only once and both clients receive the same result. This way, if the connection is lost, the client can send a new job with the same uid and, if the job is still running, the new job will not run again. Instead the client will see the result of the job that is already running.

Problems can appear when the Gearman server dies. It is simple to get pass this by starting a secondary Gearman server on standby on another machine and connect your workers to both Gearman servers. When clients can’t connect to the primary Gearman server, they will connect to the secondary server and tasks can still be run. Another advantage appears when there are too many tasks and one server can’t handle all of them. You can start another server and put clients to send tasks randomly to one server from the list.

If one of your worker machines dies or worker daemon crashes the task is not lost. The Gearman server will notice that and will give the task to another worker. So, with multiple worker machines you may rest assured that there would always be someone to pick up a task.

Gearman features

Using gearman can make your life easier. Gearman offers a lot of features. You can find more about features in the documentation. Some features are persistent queues, background tasks, priority tasks. You can check the job status and progress, check running tasks, raise the number of tasks that Gearman can handle as the number of users increases.

Another advantage is that a client and its workers can be written in different languages. For example, if you wrote a worker in php and is too slow, you can always write a faster one in C or Python and replace the slow worker in seconds. Mapreduce could also be implemented using Gearman. Although for bigger systems you would use something like Hadoop, for smaller ones Gearman could handle the network errors for you.

This way we can distribute high resource consuming task on multiple servers without having to worry about network problems and communication and, we can build a scalable and failure resistant system. When the number of users increases, the number of Gearmand servers and the number of workers can be increased to keep a low response time.

When testing a Gearman system in real world, one is not disappointed. Gearman handles the hardest part – failover, and manages workers very well. At the present time workers are running for more than one month and the failures are handled as expected. Simple network congestion are not killing the tasks because Gearman re-establishes the connections, thus making time to improve other parts of the application.

Tools

Gearman Manager is a tool for pear and pecl php libraries to launch and restart workers automatically; it runs multiple processes and allows you to have a general configuration for all workers, but also special configuration for each worker type.

Gearman Monitor is a web application written in php and you can use it to monitor one or more job servers. You can see the number of workers connected for each job, the number of waiting and running jobs.

Cons & Risks

Like any tool, Gearman is not perfect yet. Here are a few problems that might occur:

• A job might INSERT a record INTO one table and UPDATE another – increasing the row count, then crash. The job will run again and will do these queries again.
• Lack of documentation is a problem for job server, but you might find more information using -h option. Some features are undocumented at this time, like keep alive option on gearman sockets.
• No security: at this time, Gearman has no security implemented. This means that anyone can connect to the server and ask for jobs. A firewall rule should be added to prevent this from happening.