Serverless Workers & Containers
Docker in Production — What We’ve Learned Launching Over 300 Million Containers
Docker in Production at Iron.io Overview Earlier this year, we made a decision to run every task on IronWorker inside its own Docker container. Since then, we’ve run over 300,000,000 programs inside of their own private Docker containers on cloud infrastructure. Now that we’ve been in production for several months, we…
Read MoreHow to Build an ETL Pipeline for ElasticSearch Using Segment and Iron.io
Overview ETL is a common pattern in the big data world for collecting and consolidating data for storage and/or analysis. Here’s the basic process: Extract data from a variety of sources Transform data through a set of custom processes Load data to external databases or data warehouses Segment + Iron.io + Elasticsearch = A…
Read MoreOrchestrating PHP Dependencies with Composer and IronWorker
Package your dependencies on IronWorker using composer Overview This is a tutorial describing how to include and use the PHP package management tool Composer with IronWorker. Composer is a tool for dependency management in PHP. It allows you to declare the dependent libraries your project needs, and it will install them in your project for…
Read MoreIron.io Launches Custom Runtime Environments for IronWorker
Overview Iron.io announced today the introduction of custom language environments within its IronWorker processing platform. Instead of a single standard environment, developers can now define runtime environments and write workers for specific language versions. IronWorker already supports all common languages including PHP, Ruby, Python, .NET, Node.js, Java, Go, and binary files, but this release adds…
Read MoreReal-time Logging for IronWorker
Overview One of the most common feature requests we get for IronWorker is the ability to view logs in real-time while your workers are running. Until now, you had to wait for your IronWorker tasks to finish before being able to view the log and it can be pretty frustrating to try to debug your workers…
Read MoreNew Feature: Auto Retry for IronWorker
Overview IronWorker now has an auto-retry feature that, if enabled, will automatically retry your tasks if they error out. You can set the number of times it should retry and the delay between retries. Usage While uploading your worker, simply set the retries and retries-delay parameters: iron_worker upload my_worker –retries 5 –retries-delay 10 That example…
Read MoreIronWorker’s Most Requested Feature is Here: Max Concurrency
Overview Too many lemmings and you’ve got a problem. Being able to limit how many worker tasks can run at once has been one of the most requested features of IronWorker and it is available now! There are many reasons this feature is so important, here are a few of the more common use cases…
Read MoreRate Limiting Your Workers to Play Nicer with Other API’s
Overview Many third-party API’s have rate limits to ensure you don’t overuse or DoS them. Twitter for instance has the following rate limits: Unauthenticated calls are permitted 150 requests per hour. Unauthenticated calls are measured against the public-facing IP of the server or device making the request. OAuth calls are permitted 350 requests per…
Read MoreIronWorker Goes Multi-Language! Now Supports Ruby, PHP, and Python
Today marks a big milestone in the evolution if IronWorker: multi-language support. In addition to Ruby which we’ve supported from day one, we now support PHP and Python with more languages on the way. Which means if you’re using any one of those languages, you now have easy access to massive computing power. (boom) This new…
Read MoreIronWorker has a Shiny New Gem and a Shiny New API
Version 2 of the SimpleWorker API is now available featuring many new functions, a bunch of new documentation, and not to mention it is a LOT faster and more robust as it is has been rewritten from the ground up. And all this comes along with a shiny new Ruby gem. The new gem uses…
Read More