Hybrid Iron.io – On-Premise Job Processing with the Help of the Cloud

Hybrid_IronioHybridOne of our main goals for the Iron.io platform is run anywhere. This means we enable customers to use our services on any cloud, public or private. With Hybrid Iron.io, we’re making it drop dead simple to get the benefits of the public cloud, with the security and control of a private cloud. 

Using Iron.io’s public cloud service is easy, you just sign up and start using it. No servers to deal with, no setup and no maintenance. You can be up and running with a very powerful technology in a matter of minutes. It’s a beautiful thing.

On the other hand, on-premise products are a whole different ballgame. There are servers to deal with, there is a lot of setup and then there is maintenance that will continue forever. You will probably have to hire people just to maintain it. Think of a big Oracle database installation. You pay a lot upfront for the software, you pay upfront for the hardware, you pay upfront to set it up, then you continue to pay license fees, support fees and time, for as long as you use that product.

It’s not all bad though. On the plus side, you have full control of things. You choose and own the hardware it runs on, you choose where it runs, you can fix things as quickly (or as slowly) as you want, you don’t depend on a third party to run a critical part of your system, and you control the security of your data. In fact, this choice and control is really the only reason you’d ever bother with all the costs, time, and headaches involved with running something yourself.

But, what if you could have the best of both worlds? A product that has all the benefits of a managed cloud service with the control and security of an on-premise product. I’m happy to say that is now available with Hybrid Iron.io.

Hybrid Iron.io

With Hybrid Iron.io, the API and all the complexity of our job processing system lives in the cloud, while the actual execution of the workloads is on-premise, behind your firewall, on your hardware (or in your own VPC). The only thing you need to run on your systems is our runner container; no databases to install and maintain, no API servers, or anything else. The runner container talks to the Iron.io API, asking for jobs, executing them, and dealing with all the things that can happen while running. 

Iron.io Hybrid Diagram
Choice and control with Hybrid Iron.io

From a developer’s perspective, with one API, they can route jobs to any cluster whether it’s on a public cloud or on-premise just by passing in a simple parameter while queuing a job. This routing provides unmatched flexibility for distributed, high-scale job processing. Just as how containers bring a higher abstraction level between server and application environment thereby providing greater agility, hybrid runners provide a similar abstraction between clouds for your workloads. You decide where it can run, what to do on failures, what resources it needs, and many other workload-related details, but you don’t need to think about the infrastructure/hardware it runs on. #serverless

In addition, you can still use the Iron.io dashboard and see all the details and analytics of your jobs, no matter where they execute.

 

job status
Job status in Iron.io Dashboard

Security

Security is obviously a big concern here. With a full on-premise deployment, the entire system is behind the firewall and you can lock it down however you want. When we created our hybrid solution, we wanted to provide a very simple way to deploy a complex system and make it just as secure. Simple and secure.

Here’s how we make and keep things secure when using Hybrid Iron.io:

  1. All job payloads (messages) are encrypted behind the firewall, before leaving your network, with a combination of RSA public key encryption and AES-GCM encryption. Nobody can access these messages, including Iron.io.
  2. All payloads/messages are decrypted immediately before executing a job and this is done on your hardware.
  3. Job payloads/messages also have a limited data retention ranging from immediately after job completion to some time in the future.
  4. Your worker code is packaged in Docker images so if your code must also be secured beyond a private repository on a public registry like Docker Hub, you can run a private Docker registry behind the firewall. The registry location is a transparent setting in the runners. Using a private registry means your code never leaves your network and no one else has visibility into code at rest or at runtime.

Using these security measures ensures your private data never leaves your network unencrypted.

AutoScale

If you use Hybrid Iron.io on public clouds or any cloud with elasticity (ie: has an API that can launch and terminate servers on demand), then you can also take advantage of Iron.io’s autoscale capabilities.

Autoscale will automatically determine your resource usage and queue depth for your jobs, automatically launching new servers and setting them up to immediately start processing jobs. And vice versa, if usage drops, it will terminate servers automatically too. For instance, let’s say most of your jobs/workloads run during the day, Iron.io’s autoscale capabilities will ensure you have enough capacity to run during the day, then automatically shut down servers in the evening as the demand decreases. This can have a HUGE cost savings without you having to lift a finger.

Iron.io‘s autoscaling currently works with AWS, Azure, Rackspace and Google Cloud. We plan to support private cloud technologies in the future to autoscale using OpenStack, CloudFoundry, Kubernetes, etc.

Getting Started is Easy

We’ve gone to great lengths to hide the complexity of running Hybrid Iron.io and to make it as simple as possible to use. The only thing you need installed on your servers is Docker and there are only a few steps required to setup a hybrid cluster, you can even try it out on your laptop if you want.

  1. Create a cluster via our API or the Iron.io Dashboard (under Settings).
    • You will be given a cluster ID and a cluster token for authentication.
  2. Launch our iron/runner Docker image on any number of machines, passing in the cluster ID and token. The more containers you launch, the merrier.
    • The runners will communicate with the Iron.io API in the cloud and start receiving jobs to execute.
    • When you start these containers, you can set an encryption key to decrypt payloads. The runner will automatically decrypt the payloads so the people writing your workers don’t need to think about it. (See Security section above).
  3. Start queuing jobs!

That’s it. No strings. Nothing up the sleeve. It’s so easy to setup, you might think “how could this possibly be enterprise software?” It’s deceptively simple. There’s a lot going on behind the scenes, but you don’t need to deal with it.

Here’s a video demo showing how to setup a hybrid cluster and executing jobs, in a matter of minutes.

 

 

 

Conclusion

Hybrid cloud is here and we’ve tried to make it as easy and seamless as possible. Directing workloads to run on-premise or on the public cloud has never been easier. 

If you’d like to try out Hybrid Iron.io, please contact us.