Rake tasks within Ruby on Rails are compelling. It doesn’t matter if you need to do some maintenance, cleanup or complex data processing, rake tasks will emerge in your system as it is becoming bigger and more complex. More importantly, you certainly want to execute these tasks regularly and in the background.
cron is a wonderful tool to execute regular tasks in the background in a Unix-based system. However, reading and editing the schedule by using crontab can be very tedious sometimes. This will become even more true when using specific time constraints like every day at 4:00 am or every three minutes.
Thanks to the ruby gem whenever, creating such tasks will be easy as pie and using its DSL (Domain Specific Language) makes reading and editing the schedule like a charm. Installing the gem and creating your first schedule will be done in less than a minute. Add
gem 'whenever', require: falseto your Gemfile and execute a
bundle install . After that, run
bundle exec wheneverize . and you will get a
config/schedule.rb stub you can start at.
You can define a background job by simply using the keyword
every, a time constraint (e.g.
5.minutes, etc.) and a concrete job definition within the block. There are three built-in job types available:
runner… running a simple ruby command e.g. calling a service class
command… running an OS-level command
rake… running a rake task
Also, there exists the possibility to create custom job types, which will become handy later on. For simplicity, we add a basic schedule definition:
bundle exec whenever you will get the background jobs as a crontab definition shown below printed in your CLI. Adding the option
--update-crontab on the command would immediately update your crontab on your local system.
0 4 * * * /bin/bash -l -c 'cd /tmp/example && RAILS_ENV=production bundle exec rake reports:generate --silent' 0 * * * * /bin/bash -l -c 'cd /tmp/example && bundle exec bin/rails runner -e production '\''DB::Cleanup.new.call'\''' 0,5,10,15,20,25,30,35,40,45,50,55 * * * * /bin/bash -l -c 'cd /tmp/example && RAILS_ENV=production bundle exec rake data:process --silent'
Doing this on your production system would create the cron jobs for you and keep them up to date on each deployment. Finally, the
Mutual exclusion with flock
Sometimes, executing processing tasks in the background in small intervals can lead to blocking situations. To avoid this, it would be handy not to start the background job if it is still running. A mutex (in our case, a file lock) will help overcome this problem.
The linux command line tool flock enables us to manage file locks within shell scripts. It simply creates a file and sets an advisory lock for it so that another process trying to get a lock on that file will fail. As mentioned above, whenever allows us to create a custom job type. We will use a custom job type to incorporate the
flock command before executing e.g. the rake command.
Below you will find the extended example schedule from above with such a custom job type. We use the command itself and a prefix to create a unique file name for the lock file. The
flock command requires a file path and a command to execute. We can further specify the option
-n to make it non-blocking, and the execution will fail immediately if the file is locked. Using the option
-w 10 would wait for further 10 seconds until it fails.
bundle exec whenever again, the final output for our crontab will look as stated below. It creates a unique name for the lock file and uses the
flock command to prohibit multiple executions.
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /bin/bash -l -c '/usr/bin/flock -n /tmp/example/tmp/8480f0bad19e3d25b66eca0e56d2f6205581304dd589a23f8fdce6b627805365.lock -c "cd /tmp/example && bundle exec rails data:process --silent "'
wrap_with_lock method accepts a
prefix parameter, which easily enables you to create other custom jobs with a corresponding file lock. Hopefully, this will help you in some situations in your Ruby on Rails projects.