Rake tasks within Ruby on Rails are compelling. It doesn’t matter if you need to do some maintenance, cleanup or complex data processing, rake tasks will emerge in your system as it is becoming bigger and more complex. More importantly, you certainly want to execute these tasks regularly and in the background.
cron is a wonderful tool to execute regular tasks in the background in a Unix-based system. However, reading and editing the schedule by using crontab can be very tedious sometimes. This will become even more true when using specific time constraints like every day at 4:00 am or every three minutes.
Whenever
Thanks to the ruby gem whenever, creating such tasks will be easy as pie and using its DSL (Domain Specific Language) makes reading and editing the schedule like a charm. Installing the gem and creating your first schedule will be done in less than a minute. Add gem 'whenever', require: false
to your Gemfile and execute a bundle install
. After that, run bundle exec wheneverize .
and you will get a config/schedule.rb
stub you can start at.
You can define a background job by simply using the keyword every
, a time constraint (e.g. 2.hours
, 5.minutes
, etc.) and a concrete job definition within the block. There are three built-in job types available:
runner
… running a simple ruby command e.g. calling a service classcommand
… running an OS-level commandrake
… running a rake task
Also, there exists the possibility to create custom job types, which will become handy later on. For simplicity, we add a basic schedule definition:
When running bundle exec whenever
you will get the background jobs as a crontab definition shown below printed in your CLI. Adding the option --update-crontab
on the command would immediately update your crontab on your local system.
0 4 * * * /bin/bash -l -c 'cd /tmp/example && RAILS_ENV=production bundle exec rake reports:generate --silent'
0 * * * * /bin/bash -l -c 'cd /tmp/example && bundle exec bin/rails runner -e production '\''DB::Cleanup.new.call'\'''
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /bin/bash -l -c 'cd /tmp/example && RAILS_ENV=production bundle exec rake data:process --silent'
Doing this on your production system would create the cron jobs for you and keep them up to date on each deployment. Finally, the
Mutual exclusion with flock
Sometimes, executing processing tasks in the background in small intervals can lead to blocking situations. To avoid this, it would be handy not to start the background job if it is still running. A mutex (in our case, a file lock) will help overcome this problem.
The linux command line tool flock enables us to manage file locks within shell scripts. It simply creates a file and sets an advisory lock for it so that another process trying to get a lock on that file will fail. As mentioned above, whenever allows us to create a custom job type. We will use a custom job type to incorporate the flock
command before executing e.g. the rake command.
Below you will find the extended example schedule from above with such a custom job type. We use the command itself and a prefix to create a unique file name for the lock file. The flock
command requires a file path and a command to execute. We can further specify the option -n
to make it non-blocking, and the execution will fail immediately if the file is locked. Using the option -w 10
would wait for further 10 seconds until it fails.
After running bundle exec whenever
again, the final output for our crontab will look as stated below. It creates a unique name for the lock file and uses the flock
command to prohibit multiple executions.
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /bin/bash -l -c '/usr/bin/flock -n /tmp/example/tmp/8480f0bad19e3d25b66eca0e56d2f6205581304dd589a23f8fdce6b627805365.lock -c "cd /tmp/example && bundle exec rails data:process --silent "'
Our wrap_with_lock
method accepts a prefix
parameter, which easily enables you to create other custom jobs with a corresponding file lock. Hopefully, this will help you in some situations in your Ruby on Rails projects.
Leave a Reply