Chad Arimura will be speaking at the MongoSF conference next Tuesday, May 24th. He’ll be showing off the combination of SimpleWorker and MongoDB. As databases moves to more elastic and sharded NoSQL solutions like MongoDB, more and more processing moves from the data tier to the application tier. This shift makes background processing queues an even more essential component to the modern web stack.
Continue reading “Workers + NoSQL”
We’ve completed a major overhaul of the SimpleWorker interface and it is now live. The new interface is reorganized and redesigned for a much better user experience allowing you to manage and monitor your workers/jobs more effectively. Here are some sample screens:
Continue reading “Shiny New User Interface Now Live”
Performance is one of our primary goals for SimpleWorker and we’re happy to announce that we pushed out some major infrastructural changes last night that show some huge improvements in run times for all workers across the board. Here are before/after results for a particular customer’s hourly usage.
Continue reading “More Power to the Workers! Huge Performance Upgrade Went Out Last Night”
Ask And Ye Shall Receive… gems that is. The `merge_gem` feature is now in production and ready for use allowing you to use any gem you want. It will use the gems on your local file system so whatever you have available locally you can use.
It’s very easy to use:
Note: If the gem requires some native extensions, then this probably won’t work for that gem so just contact us and let us know which ones you need so we can have them installed on the SimpleWorker system.
Anyone working on a serious web app knows that a worker queue makes up a key component within the app architecture. So important is it that the infrastructure equation is typically broken into servers, workers, and datastores. Sure, there are other components and line items (cdn, bandwidth, etc) that an app can’t do within, but servers, worker, and datastores form the primary building blocks for architects and make up the main cost factors.
Continue reading “Worker Queues as a Key Variable”
Now you can start using SimpleWorker in your Heroku apps with minimal effort. It’s currently in private beta as an add-on, but let us know if you want to try it and we’ll try to get you on the list.
SimpleWorker is a job queuing and scheduling system so while a lot of work comes in at random times usually based on some event in their system (ie: a user clicks a button), there are a lot of scheduled jobs too and those scheduled jobs have a huge affect on the capacity requirements for SimpleWorker. Check out this graph of CPU usage for the past 24 hours across the entire SimpleWorker server farm.
Previously, your worker file/class had to be self contained (ie: single file including all code for the worker).
Now you can use the new “merge” class method for instance:
class TestWorker2 < SimpleWorker::Base
Continue reading “Including other files in your Worker with “merge””
After using SimpleWorkr for a long time now, I always think to myself, “there’s gotta be some way to make this more intuitive”. I want it to feel more like it’s just like running your normal code without even really realizing that the work is actually offloaded to the cloud. So with that in mind, the gem now has a much more intuitive usage pattern, more like ActiveRecord.
Continue reading “Pushed 0.3.0 Gem: Much More User Friendly”