Blog

Search This Blog

Loading...

Thursday, March 10, 2011

Why IronWorker is Better than Heroku Workers / Delayed Job

First off, I would like to say that we use and love Heroku, the system they've built is game changing and 100% awesome, but their worker system is limited and does not meet our needs. Here's the top 4 reasons why IronWorker is better than Heroku's worker system and Delayed Job (Heroku's system is based on Delayed Job). 

1. Cheaper

Heroku charges you whether you are using it or not. If you only run 10 minutes of jobs an hour, you are still paying for the full hour. With IronWorker, you are only paying for 10 minutes since you are charged by the second rather than the hour.

Price comparison:

If you run jobs that use up 10 minutes every hour, Heroku would cost $0.05 (charge by the hour) vs less than $0.01 per hour on IronWorker.  If you run jobs continuously for a solid hour, the price would be the same. 

2. Elastic, Scalable and Massively Parallel

If you run a lot of jobs, the only way to get through your job queue quickly on Heroku is to add more worker processes. If you crank it up to the max, you can get 24 worker processors meaning 24 concurrent jobs running. With IronWorker you can throw as many jobs as you want at it and they will all get run in parallel. Need to run 1000 jobs?  No problem, just queue them up in IronWorker. 

Time and price comparison:

Let's say you have 1000 jobs that you need to run every hour (for instance you need to update something for each of your users, one job per user) and let's say each job takes 30 seconds to complete. With Heroku, even using the maximum amount of workers, it would take 21 minutes to complete all jobs. With IronWorker it would take about 30 seconds to complete all jobs.

The cost for the above on Heroku would be $864 per month ($1.20 per hour, $28.80 per day). The cost for IronWorker would be $300 per month ($0.42 per hour, $10 per day).


3. Advanced Scheduling

Heroku does have scheduling capabilities via the Cron add-on, but it is very limited and only supports a single action (it call call rake cron in your application Rakefile). It also allows only two scheduling options -- hourly or daily. It does not have options to run one-time, to use different time schedules (every 15 minutes, twice at day, weekly, etc), or support multiple worker schedules. On top of this, there is also a monthly fee for the scheduling option, $3 per month.

IronWorker scheduling accommodates very flexible scheduling options and has no limit on the number of workers or schedules. Best of all, scheduling is 100% free. No additional fee.
  • Run a job just one time at some time in the future
  • Set a recurring schedule to run at any frequency you want (hourly, daily, monthly, every minute, every 15 minutes, whatever you want) 
  • Schedule any number of workers, each with their own schedules
  • Schedule jobs just as easily as you would queue jobs. Instead of the 'queue' command, you use the 'schedule' command and pass in scheduling parameters. It calls the workers directly that you schedule at the times and schedule you set.

4. Monitoring, visualizing and control

Heroku does not provide any view into your workers. With IronWorker you can:
  • View your usage patterns/trends over time
  • Check status of all your workers
  • Get notified when your workers raise errors and get down to the root of the issue fast
  • Cancel queued or scheduled jobs, kill running jobs, rerun a job




NOTES:
Calculations:
10,000 jobs at 10 seconds per job:
10000 jobs/24 worker processes = 417 jobs / worker
417 jobs * 10 seconds / job = 4170 seconds = 69.5 minutes = 1 hour and 10 minutes

1000 jobs/24 workers processes = 41.7 jobs /worker
41.7 jobs * 30 seconds / job = 1250 seconds = 21 minutes