Running Rake Tasks with Cron and File Locks

Rake tasks within Ruby on Rails are compelling. It doesn’t matter if you need to do some maintenance, cleanup or complex data processing, rake tasks will emerge in your system as it is becoming bigger and more complex. More importantly, you certainly want to execute these tasks regularly and in the background.

cron is a wonderful tool to execute regular tasks in the background in a Unix-based system. However, reading and editing the schedule by using crontab can be very tedious sometimes. This will become even more true when using specific time constraints like every day at 4:00 am or every three minutes.

Whenever

Thanks to the ruby gem whenever, creating such tasks will be easy as pie and using its DSL (Domain Specific Language) makes reading and editing the schedule like a charm. Installing the gem and creating your first schedule will be done in less than a minute. Add gem 'whenever', require: falseto your Gemfile and execute a bundle install . After that, run bundle exec wheneverize . and you will get a config/schedule.rb stub you can start at.

You can define a background job by simply using the keyword every, a time constraint (e.g. 2.hours5.minutes, etc.) and a concrete job definition within the block. There are three built-in job types available:

  • runner … running a simple ruby command e.g. calling a service class
  • command … running an OS-level command
  • rake … running a rake task

Also, there exists the possibility to create custom job types, which will become handy later on. For simplicity, we add a basic schedule definition:

When running bundle exec whenever you will get the background jobs as a crontab definition shown below printed in your CLI. Adding the option --update-crontab on the command would immediately update your crontab on your local system.

0 4 * * * /bin/bash -l -c 'cd /tmp/example && RAILS_ENV=production bundle exec rake reports:generate --silent'

0 * * * * /bin/bash -l -c 'cd /tmp/example && bundle exec bin/rails runner -e production '\''DB::Cleanup.new.call'\'''

0,5,10,15,20,25,30,35,40,45,50,55 * * * * /bin/bash -l -c 'cd /tmp/example && RAILS_ENV=production bundle exec rake data:process --silent'

Doing this on your production system would create the cron jobs for you and keep them up to date on each deployment. Finally, the

Mutual exclusion with flock

Sometimes, executing processing tasks in the background in small intervals can lead to blocking situations. To avoid this, it would be handy not to start the background job if it is still running. A mutex (in our case, a file lock) will help overcome this problem.

The linux command line tool flock enables us to manage file locks within shell scripts. It simply creates a file and sets an advisory lock for it so that another process trying to get a lock on that file will fail. As mentioned above, whenever allows us to create a custom job type. We will use a custom job type to incorporate the flock command before executing e.g. the rake command.

Below you will find the extended example schedule from above with such a custom job type. We use the command itself and a prefix to create a unique file name for the lock file. The flock command requires a file path and a command to execute. We can further specify the option -n to make it non-blocking, and the execution will fail immediately if the file is locked. Using the option -w 10 would wait for further 10 seconds until it fails.

After running bundle exec whenever again, the final output for our crontab will look as stated below. It creates a unique name for the lock file and uses the flock command to prohibit multiple executions.

0,5,10,15,20,25,30,35,40,45,50,55 * * * * /bin/bash -l -c '/usr/bin/flock -n /tmp/example/tmp/8480f0bad19e3d25b66eca0e56d2f6205581304dd589a23f8fdce6b627805365.lock -c "cd /tmp/example && bundle exec rails data:process --silent "'

Our wrap_with_lock method accepts a prefix parameter, which easily enables you to create other custom jobs with a corresponding file lock. Hopefully, this will help you in some situations in your Ruby on Rails projects.

Leave a Reply

Your email address will not be published. Required fields are marked *