Workers + NoSQL


Chad Arimura will be speaking at the MongoSF conference next Tuesday, May 24th. He’ll be showing off the combination of SimpleWorker and MongoDB. As databases moves to more elastic and sharded NoSQL solutions like MongoDB, more and more processing moves from the data tier to the application tier. This shift makes background processing queues an even more essential component to the modern web stack.
Continue reading “Workers + NoSQL”

More Power to the Workers! Huge Performance Upgrade Went Out Last Night

Performance is one of our primary goals for SimpleWorker and we’re happy to announce that we pushed out some major infrastructural changes last night that show some huge improvements in run times for all workers across the board. Here are before/after results for a particular customer’s hourly usage.
Continue reading “More Power to the Workers! Huge Performance Upgrade Went Out Last Night”

Use Any Gem You Want!

Ask And Ye Shall Receive… gems that is. The `merge_gem` feature is now in production and ready for use allowing you to use any gem you want. It will use the gems on your local file system so whatever you have available locally you can use.

It’s very easy to use:

More info.

Note: If the gem requires some native extensions, then this probably won’t work for that gem so just contact us and let us know which ones you need so we can have them installed on the SimpleWorker system.

Why IronWorker is Better than Heroku Workers / Delayed Job

First off, I would like to say that we use and love Heroku, the system they’ve built is game changing and 100% awesome, but their worker system is limited and does not meet our needs. Here’s the top 4 reasons why IronWorker is better than Heroku’s worker system and Delayed Job (Heroku’s system is based on Delayed Job).

Continue reading “Why IronWorker is Better than Heroku Workers / Delayed Job”

Worker Queues as a Key Variable

Anyone working on a serious web app knows that a worker queue makes up a key component within the app architecture. So important is it that the infrastructure equation is typically broken into servers, workers, and datastores. Sure, there are other components and line items (cdn, bandwidth, etc) that an app can’t do within, but servers, worker, and datastores form the primary building blocks for architects and make up the main cost factors.
Continue reading “Worker Queues as a Key Variable”

SimpleWorker Usage Pattern Graph

SimpleWorker is a job queuing and scheduling system so while a lot of work comes in at random times usually based on some event in their system (ie: a user clicks a button), there are a lot of scheduled jobs too and those scheduled jobs have a huge affect on the capacity requirements for SimpleWorker. Check out this graph of CPU usage for the past 24 hours across the entire SimpleWorker server farm.

Continue reading “SimpleWorker Usage Pattern Graph”

Including other files in your Worker with “merge”

Previously, your worker file/class had to be self contained (ie: single file including all code for the worker).

Now you can use the new “merge” class method for instance:

class TestWorker2 < SimpleWorker::Base

merge "models/model_1.rb"

Continue reading “Including other files in your Worker with “merge””

Pushed 0.3.0 Gem: Much More User Friendly

After using SimpleWorkr for a long time now, I always think to myself, “there’s gotta be some way to make this more intuitive”. I want it to feel more like it’s just like running your normal code without even really realizing that the work is actually offloaded to the cloud. So with that in mind, the gem now has a much more intuitive usage pattern, more like ActiveRecord.
Continue reading “Pushed 0.3.0 Gem: Much More User Friendly”