How a simple Node.js project turned into Iron Scheduler (guest post)

The Power of Task Processing in the Cloud

[David Hessing] I remember coming across Iron.io's services and immediately being intrigued. IronWorker in particular looked pretty awesome. While I had come across other cloud-based message queueing and caching tools, I hadn’t seen anything that was a pure cloud task processor. 


The idea that I could fire off independent jobs into the cloud where they would run, where I could scale them up infinitely and in parallel, and where I would never have to worry about underlying resources is incredible. 

The potential to break away from a tight app structure and move to something more distributed also appealed to me. IronWorker would force me to have good code design because my tasks had to be stateless and entirely self contained. 

Building a Flexible Scheduler in Node.js

When I started working on an independent side project, I looked at how I might leverage the Iron.io toolset. (The project at first started as a way to teach myself Node.js. Now I’m actually using it – but that’s another story.) This project needed a recurring job scheduling component and so IronWorker seemed a natural fit. 

Reading through the documentation, I came across recommendations for how to design and schedule workers. As the docs acknowledge, the built-in scheduling functionality in IronWorker is pretty basic and any complex scheduling requires custom code.

For my project, I wanted jobs to run only at selected hours of the day, and at those particular hours to run multiple times in quick bursts. I obviously needed some custom scheduler logic, and that led me to create Iron Scheduler. Iron Scheduler is simple tool that can handle some of these more complex scheduling scenarios.

How Iron Scheduler Works

At its core, Iron Scheduler is pretty straightforward – whenever it runs, it looks at a bunch of scheduled tasks that you give it. Each task has three main parameters associated to it. First a regular expression defining the schedule, second a number N defining the number of tasks to schedule and third an interval specifying the delay between each one. 

More Granular Scheduling (written in Node.js on IronWorker)

To decide whether a task should be scheduled, Iron Scheduler simply checks the regex against the current UTC time string. If it matches, Iron Scheduler will schedule the task to run N times in IronWorker, with each instance having a delay of n times the specified interval.

So for example, if the number is set to 3 and the interval is set to 20, Iron Schedule will queue three instances of the task, the first one with a delay of 0, the second with a delay of 20, and the third with a delay of 40.

Iron Scheduler can itself be run as worker on IronWorker or you can use it directly within your own Node.js projects. More detailed documentation about all the parameters and how Iron Scheduler runs is available on the project page.

Final Thoughts

It's worth mentioning that Iron Scheduler does have a few limitations, the main one being that currently there is no support for automatic timezone changes. So if say you need a job to always run at 9:00 AM standard time, you'll need to remember to adjust the scheduling regex when daylight savings rolls back around.

Iron Scheduler has helped me leverage the power of IronWorker. I hope it’s of use to others. Iron Scheduler is open sourced and so I certainly welcome enhancements and fixes!

About the Author
David Hessing is Director of Data Analytics at Appirio. He started his career designing and writing back-office trade accounting logic at Bridgewater Associates. He joined TopCoder in 2008 and eventually assumed the role of Chief Architect. (TopCoder was recently acquired by Appirio.) In addition to managing delivery on TopCoder's largest accounts, he developed and evolved many of the processes and tools that comprise the TopCoder platform. David's current  focus is on taking TopCoder's algorithm competitions to the next level — stay tuned in 2014 for amazing stuff.

–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

Part 2: Using Iron Scheduler (excerpt)

This is an excerpt from the project page on BitBucket. The scheduler can be uploaded and run as either a straight worker task within IronWorker or as an NPM (Node Packaged Module). 

{
    "ironOauthToken" : "YOUR_IRONIO_TOKEN",
    "ironProjectId" : "YOUR_IRONIO_PROJECT_ID",
    "scheduledTasks" : [
        {
            "name" : "My First Job",
            "schedule" : "b(?:13|17|21|01):.[012]:..b",
            "expires" : "Sun, 03 Nov 2013 09:00:00 GMT",
            "number" : "30",
            "interval" : "20",
            "task" : {
                "code_name" : "SomeTaskName",
                "payload" : {
                    "name0" : "value0",
                    "name1" : "value1"
                }
            }
        },
        {
            "name" : "My Second Job",
            "schedule" : "b..:(?:[03][012]|[14][567]):..b",
            "number" : "1",
            "interval" : "0",
            "task" : {
                "code_name" : "SomeOtherTaskName",
                "payloadSecret" : {
                    "name0" : "value0",
                    "name1" : "value1"
                }
            }
        }
    ]
}

Using as an Package

npm install iron-scheduler

var Scheduler = require('iron-scheduler');
var scheduler = Scheduler(ironOauthToken, ironProjectId, winstonLogger);
scheduler.run(scheduledTasks, function(err) {
    if (err) { 
        //handle error
    } else {
        //scheduler completed successfully. 
    }
});