Iron.io Releases High Memory Workers

 

IronWorker Can Now Handle Larger Tasks

We're pleased to announce the availability of high-memory workers within IronWorker. This new capability will provide greater processing power to tackle even more ambitious tasks.

The added boost to the IronWorker environment is perfect for tasks that consume large amounts of computing resources – tasks that might include big data processing, scientific analysis, image and document processing, and audio and video encoding.

A High-Performance Worker System Gets Better

Workers systems are key for scaling applications and building distributed systems – whether it’s handling tasks that get sent to the background, pre-processing data to speed up load times, scaling-out processing across large segments of data, or consuming streams of continuous events. A good worker system can handle a variety of tasks and worker patterns and address the majority of work.

A certain number of tasks, however, might not fit the typical worker system and therefore might need isolated setups with specific machine configurations. Examples might include processing large media files, doing computational analysis over large sets of data, or running other jobs that require greater machine resources, complicated code packages, or dedicated resources.

blank
Image Processing

Memory issues can be elusive to address, especially in a worker system. Depending on language, when a worker runs out of memory, it can do some strange stuff such as timeout (Node), segfault (Ruby 1.9), or just die without much indication (Ruby 2.1).

High-memory workers extend IronWorker's current capabilities so that you can pass it a greater set of application workloads. The early use cases we're seeing are for image and audio processing but you can use it for just about anything where larger in-memory resources will be a benefit.

More Memory and Faster Networking Speeds

blank
Audio Processing

The standard worker configuration provides 330 MB of memory and enough CPU power for almost all general application tasks. (This is especially true if work is split up across a number of workers and various worker patterns are employed such as master-slave or task-chaining.)

The high-memory worker configuration provides 1.5 GB+ RAM which translates into much more in-memory processing and little to no storage swapping. The high-memory workers also provide faster I/O and networking capabilities which means faster job execution, faster file transfers, and faster run times.

Getting Started with High-Memory Workers

Using our high-memory clusters is as easy as passing in a hi-mem cluster option when you queue a task. When tasks run, they'll be processed within a high-memory cluster of runners. (The feature is just starting to roll out into production so we'll need to enable your account for access.)

To get started with high-memory workers, check in with our sales team and we'll get you up and running in no time.

What are you waiting for? High-memory awaits.


To learn more about what you can do with a worker system, check out this article on top uses of IronWorker.

To try IronWorker for free, sign up for an account at Iron.io.

Leave a Comment





This site uses Akismet to reduce spam. Learn how your comment data is processed.