Search This Blog


Tuesday, August 30, 2011

Video Series - Episode 2: Scheduling

Hey All,

Here's the second video in my tutorial series. In under 3 minutes I demonstrate how to schedule a worker that posts to Twitter, and even show the code involved, which is a grand total of about 50 lines.

Posting a few tweets may not seem that impressive, but imagine if your scheduling needs scaled linearly with the number of users that sign up for your service. That's what SimpleWorker is best at - scaling your scheduling and background needs so that you don't have to.

I encourage you to subscribe to this blog and our Twitter account for a continuous feed of popular ways to utilize SimpleWorker in your business.  Once you start, it's addicting and you'll never build another background system again!

Thanks and stay tuned for the next edition of the tutorial series next week!


Thursday, August 25, 2011

Video Series - Episode 1: Getting Started

Hey All,

I'm starting a video series to help new and current customers get the most out of the SimpleWorker service. Here's the first edition, creatively titled "Getting Started".  It is a 2 minute walkthrough from signup to running workers that you can emulate exactly to help get you started using background processing as a service.

The next video, which I'll finish this week, will dive deeper into the jobs and scheduled jobs pages.

Look for these to be posted on our homepage at soon, but for now, here's a direct link:  



Friday, August 12, 2011

New Release: Job Fairness, More Generic Backend and New Gem

Hi all, we pushed out a major new release a couple of nights ago and it is a big step forward in a couple of areas:  fairness amongst all users in the system and a more generic backend to allow any Ruby code to run, more on that below.


Given a limited set of resources to be shared among a number of users, fairness is the concept of ensuring that all users get a fair share of the system and prevents heavy users from hogging all the resources.

Previously, we didn't really have provisions to handle this so when a heavy user queued up thousands of jobs, all other users would have to wait for those jobs to be run, sometimes hours. Basically a FIFO queue shared across all users. You could of course bump up the priority of your jobs to get to the head of the line, but the same problem could happen if someone queued up thousands of jobs in a higher priority.

This new release goes a long way towards fixing this issue by ensuring all users jobs will be run in a reasonable amount of time. In other words, light users will not have to wait behind a huge queue of jobs that were queued up by a heavy user.

Generic Backend and New Gem

We moved a bunch of the logic that was on the backend into the gem, such as loading other files and gems and setting up database connections. The backend will simply take what it gets and execute it (well it's not quite that simple).

Why is this good you might ask? Well there are several things:

  1. You no longer have to write your code following the same pattern we use to enforce. You could write any ruby code in any way, package it up, and as long as we can unpackage it and execute a file from the package, you're good to go. This means someone else could write a new gem that works in a different way if they'd like (challenge: feel like making something better and easier to use than our official gem??).  We'll be releasing API docs soon that will explain exactly how to do this.
  2. A lot of bugs, issues and new features can be fixed in the gem (ie: client side) which means we can fix things faster and take contributions from the community.  
  3. This leads to multi-language support, stay tuned for more on this in the near future!

Be sure to get the latest gem to take advantage of these changes, aptly versioned 1.0.X, so do a gem update or if you don't have it yet, sudo gem install simple_worker.

That's all she wrote for this round. Enjoy and let us know what you think or if you have any questions or problems.

Thursday, August 11, 2011

Heroku + SimpleWorker = General Availability

SimpleWorker is now available as General Availability in the Heroku Add-ons Catalog!

In other words, we're out of private beta and anyone can sign up for use with Heroku. W00t!

SimpleWorker is a fully featured worker system that runs elastically on the cloud. Like Delayed_job, you can pass off work asynchronously to run in the background, but with SimpleWorker you can execute hundreds of jobs in parallel!
  • Run background jobs in the cloud
  • Easy to use, little to install or setup (except your API keys)
  • Advanced scheduling - one-time in the future, recurring schedule, and more
  • Massively parallel - queue up as many jobs as you want
  • Cost effective - $0.05 per hour billed by the second so you really only pay for what you use. If you only use 1 minute per hour you only pay $0.0008 per hour.