dev2ops

Infrastucture as Code, or insights into Crowbar, Cloud Foundry, and more

Over the last few days I’ve introduced you to Crowbarand Cloud Foundry. Both are fairly new tools in the web/cloud/DevOps orbit, and worth taking a look at, or even better, getting involved in the community efforts to flesh them out into full-featured pieces of kit that are easier to use, more stable, and closer to turnkey. Are either one ready for production? Oh heck yeah, but you’ll have to be able to read the code and follow it well in order to figure out what either project’s doing under the hood: they’re both real new, and the documentation (and architecture, when it comes right down to it) are still being written.

So where are these projects going?

Let’s start with Cloud Foundry:

Cloud Foundry works, right now. It’s rough around the edges yet, and there’s still some tooling and packaging that needs to be done to make it easier to run on your own infrastructure, but the bones are very solid and they work. VMWare is actively engaging partners to help expand the capabilities of the platform. Through its Community Leads program, the project has already gained application support for Python and PHP applications.

Just yesterday, the CF project announced Cloud Foundry Micro, which is a VMWare appliance image that you can use to set up a development box on your desktop. This is a neatly packaged box that allows you to test your applications before you deploy them into the Cloud Foundry PaaS. This is cool, but a little limited: it doesn’t yet support PHP and Python, and it’s a bit of a black box. Great for developers, but if you want to crack the hood and see the shiny engine, you’ll need to roll your own.

You can do that (sort of) with the older developer instance installer, which was documented in my last article. (Link’s here for convenience.) Very soon, VMWare will be releasing more robust Chef cookbooks that hopefully will come closer to a production style install library. The cookbooks inside that install package were the basis of the pending Cloud Foundry barclamp.

Since there are now three beta PaaS products based on Cloud Foundry, the future is looking bright. It’s fully open source, and you can either use it or help build it.

So what about Crowbar?

I’m glad you asked! Crowbar is a much lower-level tool on the stack (Your users and/or customers will never see it!) and is being put together by a smaller team (for now), but it solves a very interesting problem (how do you go from a rack of powered-down servers to a running infrastructure?) and is just beginning an important shift in focus.

Crowbar, as we’ve seen before, was originally written to do one thing: install OpenStack on Dell hardware. Very soon, it will begin to install other application stacks (Cloud Foundry, to start) and is opening up to be more easily extendable. CentOS/RHEL support is in the works for client nodes (currently, only Ubuntu Maverick is supported). The initial code has been committed to enable third-party barclamps. There are a small handful of community developers, and (I’ve been assured) the door is open for more. Fork the project on github, read the docs (I’ve got some more here), and start hacking.

Bonus: I’m documenting how to create your own barclamp and the lessons I’ve learned so far. As I write this, it’s a blank page. By the time I’m done , you’ll be able to make a barclamp to deploy your own application by following my instructions.

Cloud Foundry, Crowbar, and You

This is part 2 in our ongoing series on Crowbar and the new Cloud Foundry barclamp. If you haven’t read part 1, go here. We’ll wait for you. I’ve got some nice sweet tea when you get back.

If you’ve followed the “cloud world,” you’ll know that in CloudLand, there’s three kinds of aaS. Crowbar was built as an *ahem* lever to get your IaaS in place. With the CloudFoundry barclamp, you can now begin to build a PaaS installation (Okay, this one’s brand new and has some limitations, we’ll talk about that in just a bit…) that can host web applications built on a wide variety of frameworks. (Seriously… with the latest announcement, just about any Linux/open-source based web application can be adapted to fit.)

Cloud Foundry in a Nutshell

Before we dive into running the tutorial, let’s get acquainted with Cloud Foundry and it’s architecture.

Cloud Foundry is a hosting platform for web applications. It provides runtime environments for Java, Ruby, Python, and PHP, along with a handful of dependent services like MySQL, RabbitMQ, and Redis. So you can build, say, a lightweight Sinatra app that is only an API broker to other services, or a full-blown Spring-based website with a Redis cache and MySQL back end and host them in the same hands-off (from the developer’s perspective, anyway) platform.

How does that work? Magic, my friend, magic… well, really it’s an asynchronous Rails app handling the API, talking via a NATS messaging queue to spin up the various parts for you as needed. There’s a gem you can install that’ll do all the API work for you, called vmc. You can always use the code inside the gem for some further automation if you need it, or just wrap it into your continuous integration loop to automatically deploy the latest version of your app on a successful build. (My team at DTO did it, roughly, a few weeks ago as a proof of concept)

I’ve got some links for you with much, much more in-depth information. Please wade through them at your leisure:

• Our own Cloud Foundry notes page, compiled by Alex Honor.
• In-depth discussion of the components, by Ezra Zygmuntowicz (one of the core Cloud Foundry team)
• More under-the-hood stuff on application deployment from the Cloud Foundry blog

Enough already, on with the worky bits!

I’ve put together a tutorial on how to get our own Cloud Foundry instance running on your Crowbar environment.

Now, this barclamp is in its early stages: it’s a port of the chef-solo cookbooks VMWare put together in their development environment install script… so it only supports a self-contained single box install at the moment. Work will be proceeding on shoring up the installation so that we can break out the components across multiple servers so you can create a truly dynamic environment to host just about everything. Have questions? (Yup, their cert’s bad. They know.) Want to help? Get involved!

Introduction to Crowbar (and how you can do it, too!)

Dell’s Crowbar is a provisioning tool that combines the best of Chef, OpenStack, and Convention over Configuration to enable you to easily provision and configure complex infrastructure components that would take days, if not weeks to get up and running without such a tool.

What is Crowbar?

Crowbar was oribinally written as a simplified installer for OpenStack’s compute cloud control system Nova. Because you can’t run Nova without having at least three separate components (one of which requiring its own physical server), creating an automated install system requires more than just running a script on a server or popping in a disk.

Crowbar consists of three major parts:

1. Sledgehammer: A lightweight CentOS-based boot image that does hardware discovery and phones home to Crowbar for later assignment. It can also munge the BIOS settings on Dell hardware.
2. OpenStack Manager: This is the Ruby-on-Rails based web ui to Crowbar. (Fun fact: it uses Chef as its datastore… there’s some pretty sweet Rails model hackery inside!)
3. Barclamps: The meat and potatoes of Crowbar – these are the install modules that set up and configure your infrastructure (across multiple physical servers, even)

Dell and other contributors have been working on extending Crowbar outside of just OpenStack as a generalized infrastructure provisioning tool. This article is the first in a 3-part series describing the development and build-up to a Barclamp (A crowbar install snippet) that gets CloudFoundry running for you.

So, that’s neat and all, but what if you want to hack on this stuff and don’t have a half-rack worth of servers lying around? This is DevOps, man, you virtualize it! (Automation will come later. I promise!)

Virtualizing Crowbar

The basics of this are shown on Rob Hirschfield’s blog and a little more on project wiki. Dell, being Dell, is pretty much a Windows shop. We don’t hold that against them, but the cool kids chew their DevOps chops on Mactops and Linux boxen. Since VMWare Fusion isn’t that far from VMWare Workstation, it only took a half day of reading docs and forum posts to come up with a reliable way to do it on a Mac

Those Instructions Are Here.

Bonus homework: Get that running and read down to the bottom of the DTO Knowledge Base article I just linked, and you’ll get a sneak preview of where we’re going with this article series.

Now, go on to part 2!

DevOps HackDay at VMware featuring CloudFoundry