Iron’s East/West Coast Drink-up

A bunch of Iron employees will be out and about in April, looking to meet up with customers to chat about our up and coming platform changes.  Beer (or wine, or cocktails, or <insert drink here>) will be on us! We’re sticking to the east and west coasts for now, and our current plans are:

April 5th,    San Francisco
April 14th,  Boston
April 15th,  NYC
April 17th,  Los Angeles

If you’re interested in attending, fill out the form below.  We’ll be in touch with the details once we have them confirmed on our end.  Cheers!

A Serverless Message Queue Without the Glue

More and more technologies get involved as systems grow, and it’s sometimes hard to keep track of what’s doing what. Caching layers, message queues, serverless functions, tracing frameworks… the list goes on.  Once you start sprinkling in public cloud services, you may find yourself developing your way into vendor lock-in.  All of the sudden, you’re dealing with one cloud, tons of services, and having to glue everything together in order to make the services talk to each other.  One of Iron’s primary goals is to make life easier for developers, and IronMQ’s little known “Push Queue” feature is one that can help prevent you from having to write the glue.

What are Push Queues?

IronMQ has a built-in feature called Push Queues, which when enabled, fire off an event any time a message gets pushed onto that queue. This comes in extremely handy when you immediately want to “do something” (or many things) with that message. With traditional message queues, you’d usually need to write another process that polls your queues for messages at a given duration. MQ’s push queues can fire off events to different types of endpoints, each extremely helpful in their own way.

What Type of events can be triggered?

HTTP
When a message gets put onto your push queue, IronMQ can make a POST request (with the message in the request body) to any URL of your choice. This is extremely handy when you want to notify other systems that some sort of event just happened or kick off another process.

MQ
Inception! You can have the delivery of a message populate another IronMQ queue. This is helpful if you want to tie multiple queues together or create a dead letter queue for example.

Worker
MQ can connect directly to IronWorker and pass its message as the payload to one of your jobs. How cool is that!?  In order to exemplify how cool that actually is, we’ll run through a real-life scenario.

MQ & Worker Example

Let’s say you have a time-sensitive nightly job that processes uploaded CSV files.  It needs to process all of the files uploaded during that day and finish within a set amount of time.   As your system grows and there are more CSV files to process, your nightly process starts running behind schedule.

You realize that a lot of the time spent in your nightly worker is spent formatting the CSV file into the correct format.  It would make sense to split this process into two distinct stages, formatting and processing.  When a CSV file is received, you could send a message to your push queue which in turn will kick off a “formatting” worker job to pre-process the CSV file into the correct format. Your nightly “processing” worker job will then be able to fly through the CSV files because it no longer needs to fix any formatting issues.

The beauty here is that you can continue to add more push events to the queue.  When a file is uploaded, maybe you also need to ping another worker that handles OCR or post an update to an external HTTP endpoint letting it know exactly “what” file was uploaded.  Without a push queue, you’d be adding a lot of custom code to handle these requests, retries, errors, etc.  IronMQ’s push queues take care of all of this for you.

How can I configure a Push Queue?

Retries
You can configure your queue to allow for a custom amount of retries, a custom delay between retries, and even provide another queue to store failed push attempts. For example, using an HTTP event, MQ will retry pushes (3 times by default) every time it receives a non-200 response code.

Timeouts
If your event never receives a response after a certain period of time (10 seconds by default), it will chalk that up as a failed attempt and retry.

Unicast or Multicast?
You can even fire off multiple events from one queue. If you need to trigger one HTTP endpoint and also fire off a Worker job, that’s not a problem.

How do I create a push queue?

Creating one is straightforward.  Here’s an example cURL request that creates a multicast Push Queue with an HTTP endpoint as well as a Worker endpoint.

IronMQ has client libraries available in most languages, so you can easily create one programmatically as well.  Here’s an example in PHP:

Conclusion

With one IronMQ Push Queue, you can make a lot happen. If you were to try and replicate a multicast Push Queue in a traditional message queue, for example, you’d end up writing a lot of custom code to glue everything together. You’d also have to deal with scaling your infrastructure as your message queue needs grew. With IronMQ, you can save time and money focusing on your applications business logic, and less time on glue and infrastructure. For more detailed information about Push Queues, visit the IronMQ documentation.

If you’re interested in knowing more about IronMQ, or want to chat about how we may be able to help, call us anytime at 888-501-4766 or email at support@iron.io.

* These fields are required.


Docker, Inc isn’t Dead

Chris Short recently wrote up a piece entitled Docker, Inc is Dead, with a prediction that the company would no longer exist sometime in 2018.  It’s well written and he does a good job of running through some of Docker’s history in recent years.  Although I agree with some of his sentiments, I don’t think Docker, Inc will exit the stage anytime soon.  Here are some reasons I think Docker, Inc will live a healthy life in 2018.

Docker is Good Software

This was the first point in Chris’ piece, and he’s right.  Docker definitely helped widen the spotlight on *n?x kernels.  Discussions around namespaces, cgroups, lxc, zones, jails, etc… lit up across communities in different disciplines.  Dockers’ simple interface lowered the barrier of entry for non-administrators, and the developer community immediately added it to their workflows.  Docker released EE/UCP, and larger organizations jumped on board.  It “is” good software for developers, SMB’s, and large organizations, and Docker, Inc isn’t slowing down development efforts.

DOCKER HAS FRIENDS

“I’m really excited to welcome Solomon and Docker to the Kubernetes community”.  Brendan Burns (of Microsoft, Lead Engineer of Kubernetes) definitely made me do a double take when he said that on stage at DockerCon EU a few months ago.  Many people I spoke to at the conference referenced that statement and saw this as a big blow to Docker.  “Who’s joining who’s community? ”  The thing is, the real purpose of Brendan’s talk was about the collaboration between companies, and the effort to make our lives as developers and administrators better.  The whole “it takes a village to raise a child” saying.  This village is composed of some of the brightest engineers from many of the world’s largest companies, and they’re all striving to make things better.  Docker and Kubernetes worked together, and the Kubernetes integration into UCP made perfect sense.

Docker has business

They don’t have a lack of coherent leadership.  They’ve received a ton of money, their marketing is great, and they’re acting like what they are;  a rapidly growing company moving into the enterprise market.  Were some of their keynotes awkward at DockerCon EU this year?  Yes.  Were there fantastic sessions from customers who shared real-life Docker success stories?  Yes.  Have they made some mistakes here and there?  Yes.  Have they moved past those and grown?  Yes.  If you’ve been around the block and watched small companies rapidly grow into behemoths, this is all typical.  Growing isn’t easy.  Their “Modernizing Enterprise Applications” mantra is perfect.  There are countless technical budgets from Fortune 10,000 companies that Docker, Inc will capitalize on.  The best part is that they’ll actually be making a positive difference.  They are not snake-oil salesmen.  These companies will probably see real ROI in their engagements.

Conclusion

Docker, Inc isn’t going to be acquired (yet) or close their doors.  There is a lot going on at Docker, Inc right now but they aren’t signs of a company that is getting ready for a sale.

It’s a company that’s based on OSS with a lot of opportunity in the market.  While one of the products at Iron is Docker-based, we use a wide variety of software from many companies with roots in OSS.  We’re happy to pay for a higher level of support and features for OSS software backed by a business.  For other projects, we often donate through Open Collective to help maintainers and small development teams.  Docker’s donation of containerd was a great move and I think it is a project that fits perfectly into CNCF’s charter.

While Docker, Inc is moving upstream, they haven’t at all abandoned its real users;  developers.   We use Docker daily, contribute back when we can, and are optimistic about its trajectory as a business and a product.  Docker, Inc has a lot of room to grow, and in 2018, it will.

* These fields are required.


Webhooks the Right Way™

If you’re a developer, dealing with webhooks is a part of your life. Nowadays almost every subscription service allows for these user-defined callbacks.  For example, when a Lead is added to Salesforce, you may want to have a task that runs in the background to generate more information about the company they work for.  Maybe you want to receive a request from Stripe when a customers payment fails so you can send them dunning emails?  You get the drift.

The most common way to deal with webhooks is adding an endpoint to your application that handles the request and response. There are some benefits to this.  No external dependencies by having all your code in one place for example.  However, the cons usually outweigh the pros.

Common problems handling Webhooks

Application downtime

If your application is down, or in maintenance mode, you won’t be able to accept webhooks.  Most external services will have retries built in but there are many that don’t.  You’d need to be OK with missing data sent from these services.

IronMQ and IronWorker have great uptime
If your application is down, you could lose valuable data
Request queuing

What happens if you have a ton of webhooks from a bunch of different services all coming in at once?  Your application/reverse proxy/etc will probably end up queuing up the requests along with other customer requests.  If your application is customer facing, this could result in a degraded user experience or even full-blown timeouts.

Use IronMQ and IronWorker to prevent bad user experiences
Too many requests to your frontend could result in request queuing and negatively affect the end user experience
Thundering herds and Cache stampedes

Even if you’re able to process all of the webhooks coming in at once, your system is going to feel the effects one way or another.  This could result in unwanted resource spikes (CPU/MEM/IO).  Unless you’re set up to autoscale, bad things could happen to your infrastructure.

 

IronMQ and IronWorker can help prevent downtime caused by webhooks
Handling webhooks at scale via your application could result in infrastructure issues

 

At Iron, many of our customers get around these issues by using IronMQ and IronWorker in conjunction.  Since IronMQ is HTTP based, highly available, and built to handle thousands of requests a second, it’s a perfect candidate for receiving webhooks.  One of the great things about IronMQ is that it supports push queues.  When a message is received, it can push its payload to an external HTTP endpoint, to another queue, or even to IronWorker.

IronWorker is a container based enterprise-ready background job processing system that can autoscale up and down transparently.  We have customers processing 100’s of jobs concurrently one minute, while the next minute the number is up in the 100’s of thousands.

The beauty of the IronMQ and IronWorker integration is that IronMQ can push its payloads directly to IronWorker.  Your work is then picked up and worked on immediately (or at a specific date and time if required).  You can have a suite of different workers firing off for different types of webhooks and handling this process transparently.  This is great for a number of reasons.

Handling Webhooks the Right Way

Happy application

Now your application is never involved in the process of handling webhooks.  This all happens outside of your normal application lifecycle.  Your application machines will never have to deal with the excessive load that could deal with infrastructure issues.

Happy users

All the work you need to do to process webhooks now happens in the background and on hardware that your users aren’t interacting with.  This ensures that processing your webhooks won’t affect your user experience.

MQ and Worker to handle Webhooks
Using IronMQ and IronWorker to handle incoming Webhooks

This is a pattern that our customers are pretty happy with, and we’re constantly improving both IronMQ and IronWorker to handle their additional needs. For example, being able to programmatically validate external API signatures and the ability to respond with custom response codes are on our list.  That said, similar to microservices, this level of service abstraction can also introduce its own complexities.  For example, dependency and access management come to mind.  We’ve had long conversations about these topics with our customers and in almost all cases, the pro’s out-weigh the cons.  This approach has been a success and we’re seeing it implemented more and more.

If you have any questions or want to get started with the pattern above, contact us and we’ll be happy to help.

* These fields are required.


AWS EC2: P2 vs P3 instances

Amazon announced its latest generation of general-purpose GPU instances (P3) the other day, almost exactly a year after the launch of its first general-purpose GPU offering (P2).  While the CPU’s on both suites of instance types are similar (both Intel Broadwell Xeon’s), the GPU’s definitely improved.  Note that the P2/P3 instance types are well suited for tasks that have heavy computation needs (Machine Learning, Computational Finance, etc) and that AWS does provide G3 and EG1 instances specifically for graphic intensive applications.

The P2’s sport NVIDIA GK210 GPU’s whereas the P3’s run NVIDIA Tesla V100’s.  Without digging too deep into the GPU internals, the Tesla V100’s are a huge leap forward in design and specifically target the needs of those running computationally intensive machine learning operations.  Tesla V100’s tout “Tensor cores” which increase the performance of floating point computations and the larger of the P3 instance types support NVIDIA’s “NVLINK”, which allow multiple GPU’s to share intermediate results at high speeds.

While the P3’s are more expensive than the P2’s, they fill in the large gaps in on-demand pricing that existed when just the P2’s were available.  That said, if you’re running a ton of heavy GPU computation through EC2, you might find the P3’s that offer NVLink a better fit, and picking them up off the spot market might make a lot of sense (they’re quite expensive).

When the P2’s first came out, Iraklis Mathiopoulos had a great blog post where he ran Hashcat (a popular password “recovery” tool) with GPU support against the largest instance size available… the p2.16xlarge.  Just a few days ago he repeated the test against the largest of the P3 instances, the p3.16xlarge.  If you’ve ever played around with Hashcat on your local machine, you’ll quickly realize how insanely fast one p3.16xlarge can compute.  Iraklis’ test on the p2.16xlarge cranked out 12,275.6 MH/s (million hashes per second) while the p3.16xlarge at 59,971.8 MH/s against SHA-256.  The author’s late 2013 MBP clocks in at a whopping 121.7 MH/s.  The p3.16xlarge instance type is about to get some heavy usage by AWS customers who are concerned with results rather than price.

Of course, the test above is elementary and doesn’t exactly show the benefits on the NVIDIA Tesla V100 vs the NVIDIA GK210 in regard to ML/AI and neural network operations.  We’re currently testing different GPU’s in our Worker product and hope to have some benchmarks we can soon share based on real customer workloads in the ML/AI space.  The performance metrics and graphs that Worker produces will give a great visual on model building/teaching, and we’re excited to share our recent work with our current ML/AI customers.

While most of our ML/AI customers are on-premise, we’ll soon be looking to demonstrate Iron’s integration with P2 and P3 instances for GPU compute in public forums. In the meantime, if you are considering on-premise or hybrid solutions for ML/AI tasks, or looking to integrate the power of GPU compute, reach out and we’d be happy to help find an optimal strategy based on your needs.

Iron.io at The Machine Learning Conference

 

 

Attending MLconf in San Francisco on November 10th? If so, come say hello!

We’ve been seeing more and more customers hiring machine learning talent in order to tackle operational efficiencies and hone in on their forecasting. Iron’s platform is helping in almost all phases of the process, from ETL operations helping with the build phase to building models through distributed, hybrid and on-prem, IronWorker deployments.  We’ve never thought of ourselves as a machine learning as a service (MLaaS) company, but we’re apparently getting a lot of traction in the industry which is music to our ears!

The speakers this year are incredible and we’re looking forward to the entire event. From Xavier Amatriain’s background with ML driven medicine to Franziska Bell’s work on uncertainty estimations at Uber, we’re pretty awestruck at the lineup.

The event is being held at the Nikko Hotel in San Francisco on the 10th of November, and you can find more details here:  https://mlconf.com/events/san-francisco-ca-2/

We’ll be following up with a great recap, so stay tuned.  We hope to see you there!

GPU support in IronWorker

In the past few months, we’ve spoken to quite a few customers that have added Machine Learning (ML) tasks into IronWorker. The problem is, these tasks can take a significant amount of time on a CPU vs a GPU. GPU’s were built to handle the parallelization of complex matrix/vector operations that gaming required, and it so happens that deep learning exercises also have similar requirements.

That said, we thought it was about time to add GPU support to IronWorker. We started with a simple test of doing image recognition via TensorFlow. After hacking the example python script to add the ability to download an image via a URL, we zipped up the script and uploaded it to IronWorker. We went ahead and used the latest Tensorflow docker image.

> zip classify_image.zip classify_image.py
> iron worker upload --zip classify_image.zip --name classify_image tensorflow/tensorflow:latest-gpu python classify_image.py --image_url "https://www.petfinder.com/wp-content/uploads/2012/11/91615172-find-a-lump-on-cats-skin-632x475.jpg"

In this initial push, we released support for the g2, g3 and p2 GPU instances on AWS. Once we fired off that task, here’s what things look like on our end:

It took about 10 seconds of run time for the image classification via IronWorker using a p2.xlarge instance on AWS. We didn’t have a chance to run this against a non-GPU instance, but we’ll leave that as an exercise for the reader. We’re pretty sure it will take a little longer than 10 seconds! The actual output from the script is as follows:


Found device 0 with properties:
name: Tesla K80
major: 3 minor: 7 memoryClockRate (GHz) 0.8235
pciBusID 0000:00:1e.0
Total memory: 11.17GiB
Free memory: 11.10GiB

Egyptian cat (score = 0.60871)
tabby, tabby cat (score = 0.12714)
lynx, catamount (score = 0.07766)
tiger cat (score = 0.07641)
cougar, puma, catamount, mountain lion, painter, panther, Felis concolor (score = 0.00148)

We’ll be working closely with a few customers in the coming months on some large ML/AI projects, and we’ll post as much as we can on their use cases and resulting benchmarks. As ML becomes more and more prominent in Business Intelligence operations, we’re expecting to see a big increase in GPU usage. If you have any questions about our GPU support, drop us a line and we’d be happy to chat.

Robotics with Iron

We recently sponsored a robotics event in Tokyo held by OnLab, DigitalGarage and Psygig… and, it was awesome.  Also, yes… the course below was an actual course from the event.

The participants were to break into teams, build a drone, implement machine learning techniques, gather and analyze data via Iron, and maneuver their drone across multiple courses.  The teams that finished the courses and displayed the most innovative technical solutions were crowned champion.

 

 

The ability to utilize GPU instances and fire up containers that run libraries like Keras and TensorFlow allows for the offloading of heavy computational workloads even in highly dynamic environments.  In the last few months, we’ve been speaking to more and more customers who are using Iron for large ML and AI workloads, often breaking them into distinct types of work units that require different levels of GPU, CPU and memory requirements.

Congratulations to the winners of the contest and all those that participated!  It looked incredibly challenging.  If you have any questions about utilizing GPU’s, machine learning, artificial intelligence, or any other computational heavy lifting jobs using Iron, feel free to contact us at support@iron.io as we’ll be happy to chat.

 

 

Full Circle and Ramping up at Iron.io

Iron.io was recently acquired by Xenon Ventures, a private equity, and venture capital firm. Xenon Ventures is headed by Jonathan Siegel, a serial entrepreneur who has founded many popular software services and has made just as many successful acquisitions.

Here comes the full circle. What may not know, is that Jonathan was Iron.io’s first customer and investor back in 2010 prior to Iron.io’s creation. Jonathan was client and friend of the founders’ consulting business prior to Iron.io and encouraged the founders to transform their consulting service into a product.

In 2011 the first version of IronWorker was launched and the serverless revolution began. After pioneering this space, Iron.io has grown significantly adding products like IronMQ, IronCache, and our latest development of our Open Source product, IronFunctions. This success is due all of our amazing customers and partners! Thank You!!

New and old faces

You may be seeing a new name fly around a bit as well.  I’ll (Dylan Stamat) be joining Iron.io as General Manager and moving Iron forward. A little about me: I’ve been a personal friend to the founders, I was a co-founder at RightSignature (https://rightsignature.com), founded one of the first HIPAA compliant companies to run on AWS (https://verticalchange.com), was previously the CTO at a large technology consultancy company (ELC Technologies), a Ruby on Rails contributor, a committer to Ehcache, and a big fan of Erlang and Golang.

You will also see and hear from many other familiar faces at Iron.io – Roman Kononov, Director of Engineering who has been leading our engineering office in Bishkek, Kyrgyzstan since 2011. Nikesh Shah, Director of Business Development and Marketing, and other various new and old members from our globally distributed teams.

Roman Kononov and Rob Pike (with an awesome jacket) at Gopherfest.

As of now – we’ve added 2 new offices to Iron: one in Las Vegas, Nevada, the other in Tokyo, Japan. We’ve hired new Engineers and Customer Success and are continuing to hire. Let us know if you have any interest in joining our team!

What’s to expect moving forward?

New graphs providing quick ways to visualize historical worker concurrency

The short answer is, things are going to get a lot better.  We’ve been very busy since the acquisition.  There have been a lot of bug fixes, improvements to internal tooling, and we’ve added concurrency graphs to help provide more insight into the system.

Near term, we are committed to continuing and ramping up development in our entire product line. This includes better performance and reliability, a new user interface, granular metrics reporting such as concurrency graphs, streamlining customer support and putting new systems in place to better track feature requests and bug reports, and bug fixes throughout our web applications.

Long term, as it relates to products, we are being guided by 2 core principles:
Open Source, and Hybrid Cloud Deployments.

I’m excited about the future, and getting to know all of you! We will have more exciting news to announce in the coming months.  Please feel to reach out to us and stay tuned!

Dylan Stamat
GM, Iron.io