Search This Blog


Friday, July 15, 2011

SimpleWorker is a Citrix Accelerator Finalist!

We're proud to announce that SimpleWorker is a finalist in the Citrix Global Challenge 2011.  Hundreds of companies were vetted and distilled down to 12 finalists based on a number of criterion, a couple of which I will highlight.

Emphasis on solving business problems

Background processing is hard.  Scaling it it is even harder.  We solve that problem by leveraging economies of scale and cloud technologies so our users don't have to.  This allows SimpleWorker customers to focus on creating value for their own customers quicker without worrying about background processing.

A superior user experience

The user experience starts the second the first worker is queued and ran.  We often hear these users say "wow, and it ran just like that?".  The experience continues through our UI as you peel the layers of the onion back to reveal everything about your workers: status, logs, analytics, etc.  We'll continue to innovate and improve this experience to bring more value to the table.

We're grateful to Citrix for their validation of our mission and are super excited to continue in our goal of providing the very best background processing service.


Wednesday, July 13, 2011

Anti-Pattern: Lots of Scheduled Jobs

IronWorker provides an easy way to schedule jobs in the cloud without being tied to a particular server. Think Cron in the cloud.)

With the cloud, elasticity, and the modern app stack (servers + workers + datastores), this scheduling flexibility opens up new possibilities in terms of handling an app's processing needs. It's now possible to schedule jobs and have them coordinate running thousands of jobs and processing lots of data across a number of servers -- all without having to manage servers or infrastructure.

But with this power brings with it some good and not-so-good ways of doing things. We addressed a pattern for creating Master/Slave jobs in a blog not too long ago. We've used this approach successfully within applications we've written and have recommend it far and wide.
Over the course of the last few days, though, we've gotten a good look at the anti-pattern. What happens when you create a lot of scheduled jobs -- one for each low level task (instead of creating a master worker that creates slave workers when it comes off schedule).

Take our word for it. It's not pretty. Either from an application standpoint (keeping track of 10s of thousands of scheduled jobs) or from a job queueing/management standpoint (lots of processing to keep things on schedule).

After talking with a customer building a test app and doing a bit of work on our end, things are back to normal. There may have been some delays in scheduled jobs running last night. We apologize. We've addressed some bottlenecks and will be creating additional safeguards to make sure quality of service remains high.

Addressing this issue does, however, offer us an opportunity to provide the following lesson for app development:
Anti-Pattern: If you're scheduling lots of jobs, you're probably doing something wrong.
Instead, schedule jobs that accesses a data store and then queues up jobs to perform the work that is needed. Much easier that way.