How to Build a Serverless Search Engine with SimpleWorker and IndexTank

Our friends over at IndexTank recently posted an article on how to create a search page using IndexTank and SimpleWorker. SimpleWorker to gather data from a remote service, in this case Plixi, and IndexTank to index that data for searching. SimpleWorker is great for collecting and keeping data up to date because you can schedule a…

Read More

Pattern: Creating Task-Level Workers at Runtime

Overview A type of topic appearing more and more frequently in StackOverflow and Quora are questions on general architecture and app design. We came across one the other day on approaches to queuing and scheduling workers. A good worker pattern – based on what we’ve seen and done – is to chunk the work (and…

Read More

MongoSF Was Awesome

Had a great time at the MongoSF conference last week. Meghan Gill, Melia Jones, Max Schireson and the rest of the 10gen team put on a remarkable one-day event for all things Mongo and NoSQL. Some great companies presenting and displaying. MongoHQ, MongoLabs, DotCloud, Cloud Foundry were just a few.

Read More

Quintupled Capacity Over the Past Week

Over the past week, we have seen tremendous usage growth on SimpleWorker. Our queues were getting backed up and our poor little workers just couldn’t keep up. Well we threw some horsepower at the problem… lots of horsepower, and now we’re running smooth again. Sorry for those who had extra long wait times, we’re growing…

Read More

Workers + NoSQL

Chad Arimura will be speaking at the MongoSF conference next Tuesday, May 24th. He’ll be showing off the combination of SimpleWorker and MongoDB. As databases moves to more elastic and sharded NoSQL solutions like MongoDB, more and more processing moves from the data tier to the application tier. This shift makes background processing queues an…

Read More

Shiny New User Interface Now Live

We’ve completed a major overhaul of the SimpleWorker interface and it is now live. The new interface is reorganized and redesigned for a much better user experience allowing you to manage and monitor your workers/jobs more effectively. Here are some sample screens:

Read More