Right-sizing with Docker Stats and cAdvisor

Right-sizing Docker

Right-sizing Docker

Thanks to Jared for the base image CC BY 2.0

Containers make life easy. Oh, you don’t have Ruby 2.2 installed? No problem, try this Docker image. Knowing what I tested on my local is exactly what’s running on production gives me warm fuzzies.

Docker gets a lot of love because it simplifies development. That’s not all though. If Docker punished infrastructure, there’d probably be a lot less love going around. Thankfully, Docker does some cool things on the infrastructure side, as well.

The biggest benefit is the “right-sizing” of compute resources. Your program might only need 200 MB of memory. Why dedicate an entire VM + OS to that? Docker insures our compute resources are neatly divided by memory and CPU between instances. Neat! There’s a lot to love about Docker on the infrastructure side, as well.

To make all this magic happen, devs just need to check how resource hungry their creations are. Without a good check, a hungry program can turn production servers into a Little Shop of Horrors. “Feed me, Seymour.”

Monitoring with Docker Stats

Thankfully, the Docker toolbox comes with a lot of handy bits to help with this. Let’s take a look at the included Docker stats command.

Docker stats provides a live view of both CPU and RAM, while a container is running. To invoke it simply type docker stats <container-name-here> while a container is running. This will let us see if our code is CPU bound or memory bound. After which we can either improve the code, or increase the amount of resources allocated to the container.

In the case of the public IronWorker, we’re limited to 320MB of RAM. If you’re a paying customer, you have many more options for right-sizing your Docker containers. For the sake of this article, I’ll use the public option as my test target. Feel free to get in touch with us, if you think your code will benefit from a beefier setup.

Step 1

Clone the DockerWorker repository. If you haven’t already, head over to github.com/iron-io/dockerworker and clone the repository. For most, this is as simple as:

git clone https://github.com/iron-io/dockerworker

Next, follow the instructions in the README. This will ensure you’ve installed both Docker and Iron CLI properly.

Step 2

Put hungry code in a container. I made an (ugly) solution to a Project Euler problem in Ruby. So, Ruby is going to by my target for this test. The code I’ll be running will check a large number for prime factors.

The code follows:

Navigate to the Ruby folder, and open hello.rb. Replace the contents of hello.rb with the above code and save.

The above doesn’t have any dependencies, but just in case you’re running your own hungry code, be sure vendor them with:
docker run --rm -v "$PWD":/worker -w /worker
iron/ruby:dev bundle install --standalone --clean

Step 3

Ponder the beauty of numbers, nature, and life as stats pour in. Our final step is to run our hungry code, and read the stats.

To do this, we’ll need to add two arguments to our usual Docker run command. The first is -m 320M, which sets the maximum allowed memory. The second is using –name to give our container a name.
docker run --name lizLemon -m 320M --rm -it
-e "PAYLOAD_FILE=hello.payload.json" -v "$PWD":/worker -w /worker
iron/ruby ruby hello.rb

I named my container lizLemon. I find the best names are ones that bring the most joy while cursing loudly. “Damnit, Lemon!”

Next, open a second terminal window. Make sure it’s capable of running a docker command by typing:
docker run hello-world

If you see a friendly message, you’re good to go. In window #1 run the above docker command. While that’s happening, quickly head to window #2 and enter:
docker stats lizLemon

On success, you should see something along the lines of:

docker stats command in action

As expected, code for decomposing primes is CPU bound. Yay!

Monitoring with cAdvisor

Switching between between terminals is a bit janky. To avoid the frantic back and forth, we can look to another solution: cAdvisor.

cAdvisor is a web UI, built by Google. It’s goal is to make monitoring container resources simple. It runs as it’s own Docker image in the background, and on OSX it’ll launch from the following command:
docker run
--volume=/:/rootfs:ro
--volume=/var/run:/var/run:rw
--volume=/sys:/sys:ro
--volume=/var/lib/docker/:/var/lib/docker:ro
--publish=8080:8080
--detach=true
--name=cadvisor
google/cadvisor:latest

If you’re on another *nix variant, you’ll likely need to prepend a ‘sudo’ to the command.

For OSX users the next step is finding the local IP. This can be done with:
echo $DOCKER_HOST

Copy the IP and add the port :8080. Paste the result into a web browser. On *nix variants you should be able to just head to localhost:8080. You should see a page that looks like:
The cAdvisor webUI

Now, run the lizLemon (docker run –name lizLemon…etc) you’ll find very detailed results in cAdvisor.

Hardware is Expensive

Hardware is expensive! Improving hardware utilization by even a meager 10% can take a huge bite out of your monthly bill.

As shown above, testing the “hunger” of code is relatively simple. It’s a low risk, and high reward situation. So, no excuses! Talk with the team, and insure you’re right-sizing your Docker containers.