ETL Archives - The Iron.io Blog

Run ETL using Background Jobs Solution: A Hybrid Model

Korak Bhaduri — Fri, 21 Jul 2023 11:54:47 +0000

If you are a developer dealing with the complex process of data management, particularly with ETL (Extract, Transform, Load), this article is for you. We will guide you through the cases where running ETL jobs using background job solutions in a hybrid (or on-premise) model can be useful. Additionally, you will learn how IronWorker can…

The post Run ETL using Background Jobs Solution: A Hybrid Model appeared first on The Iron.io Blog.

The E.T. in ETL

Iron — Sat, 09 Jan 2016 03:27:45 +0000

Thanks to JD Hancock for the base image! CC BY 2.0 Anyone who’s ever done ETL knows it can get seriously funky. When I first started working on ETL, I was parsing data for a real estate company. Every once in awhile roofing data would appear in the pool field. “Shingles” isn’t a compelling feature…

The post The E.T. in ETL appeared first on The Iron.io Blog.

E is for Event: A Fresh Take on ETL

Iron — Tue, 15 Sep 2015 03:31:36 +0000

As a follow up to my previous post, The Workloads of the Internet of Things, I wanted to walk through a real world example that fully captures the principles of event-driven computing put forth. Let’s set the stage first... imagine we operate a windmill farm and want to continually track optimal weather conditions to maximize…

The post E is for Event: A Fresh Take on ETL appeared first on The Iron.io Blog.

How HotelTonight Streamlined their ETL Process Using IronWorker

Iron — Wed, 08 Apr 2015 05:44:20 +0000

HotelTonight has reinvented the task of finding and booking discounted hotel rooms at travel destinations. Designed for last-minute travel planners and optimized for the mobile era, HotelTonight connects adventure-seeking, impulse travelers with just-in-time available hotel rooms wherever they land. This model has the market-enhancing effect of reducing excess inventory of unused hotel rooms, while delivering…

The post How HotelTonight Streamlined their ETL Process Using IronWorker appeared first on The Iron.io Blog.

How to Build an ETL Pipeline for ElasticSearch Using Segment and Iron.io

Iron — Thu, 02 Oct 2014 17:00:00 +0000

Overview ETL is a common pattern in the big data world for collecting and consolidating data for storage and/or analysis. Here's the basic process: Extract data from a variety of sources Transform data through a set of custom processes Load data to external databases or data warehouses Segment + Iron.io + Elasticsearch = A…

The post How to Build an ETL Pipeline for ElasticSearch Using Segment and Iron.io appeared first on The Iron.io Blog.

How HotelTonight uses Iron.io and AWS Redshift to create Ruby-based ETL pipeline (repost)

Iron — Tue, 03 Jun 2014 17:39:00 +0000

Creating an ETL pipeline with Iron.io and Redshift Operating at scale in the cloud almost always equates to having a highly distributed system architecture in place to handle workloads by auto-scaling components out horizontally Harlow Ward is a developer at HotelTonight and he put together a great post on how they handle issues of…

The post How HotelTonight uses Iron.io and AWS Redshift to create Ruby-based ETL pipeline (repost) appeared first on The Iron.io Blog.