Last Tuesday, I gave a talk at Austin on Rails about background processing options available in Ruby on Rails. The talk was divided into 3 sections.

The first, was ad hoc background processing solutions based on built in Rails tools or simple gems. These strategies tends to be good for background processing that need to run on a fixed schedule. These strategies not persistent and are memory intensive.

The second category I talked about was simple queues. There are several gems and plugins that fit this category. The jobs are primarily persisted in the database. These solutions are relatively simple to set up, typically persist to a database table or an external queue. These are great for user generated jobs that need to run asynchronously from the typical request/response cycle. I covered some of the more popular options: BackgrounDRb, Background Job, Delayed Job, Working/Starling. We have had success with Background Job and Delayed Job where I work, but the other offer unique advantages and are worth investigation. Workling not a complete background solution by itself, but an interface to background job creation and management. In addition to Workling, you also need a runner to handle the actual running/queuing of the jobs. Spawn and Starling are popular options here. Workling allows you to switch out the underlying system that manages the queue as your needs change.

The final section of the talk focused on more advanced queues. These are all external queues completely separate from your Rails application. I mentioned Amazon SQS, RabbitMQ, Kestrel as good options to explore if you are looking for high traffic background processing.