View All Videos

Archive for September, 2011

Video: Marten Mickos and Rich Wolski talk DevOps and Private Clouds

Damon Edwards / 

I ran into Marten Mickos and Rich Wolski from Eucalyptus Systems at PuppetConf and got them to sit down for a quick video alongside my fellow and DevOps Cafe contributor, John Willis.

I had just come out of Marten’s keynote where he spoke about DevOps far more than I would have expected. In this video we explore the deep connection between DevOps and Private Clouds as well as other industry changes for which they are planning.

Eucalyptus was one of the first private cloud technologies on the scene, and consequently got the benefit and burden of being the early mover. The community had some ups and downs along the way, but their product and industry vision seems encouraging and warrants a closer look (and never count out Marten Mickos in an open source software battle).

Puppet and Chef Rock. Doh. What about all these shell scripts ?!


Alex Honor / 

Incorporating a next generation CM tool like Puppet or Chef into your application or system operations is a great way to throw control around your key administrative processes.

Of course, to make the move to a new CM tool, you need to adapt your current processes into the paradigm defined by the new CM tool. There is an upfront cost to retool (and sometimes to rethink) but later on the rewards will come in the form of great time savings and consistency. 

Seems like an easy argument. Why can’t everybody just start working that way? 

If you are in a startup or a greenfield environment, it is just as simple as deciding to work that way and then individually learning some new skills.


In an enterprise or legacy environment, it is not so simple. A lot of things can get in the way and the difficulty becomes apparent when you consider that you are asking an organization to make some pretty big changes:
  • It’s something new: It’s a new tool and a new process.
  • It changes the way people work: There’s a new methodology on how one manages change through a CM process and how teams will work together.
  • Skill base not there yet: The CM model and implementation languages needs to be institutionalized across the organization.
  • It’s a strategic technology choice: To pick a CM tool or not to pick a CM tool isn’t just which one you choose (eg, puppet vs chef). It’s about committing to a new way of working and designing how infrastructure and operations are managed.
Moving to a next generation CM tool like Chef or Puppet is big decision and in organizations already at scale it usually can’t be done whole hog in one mammoth step. I’ve seen all too often where organizations realize that the move to CM is a more complicated task than they thought and subsequently procrastinate.

So what are some blocking and tackling moves you can use to make progress?

Begin by asking the question, how are these activities being done right now?
I bet you’ll find that most activities are handled by shell scripts of various sorts: old ones, well written ones, hokey rickety hairballs, true works of art. You’ll see a huge continuum of quality and style. You’ll also find lots of people very comfortable creating automation using shell scripts. Many of those people have built comfortable careers on those skills.


This brings me to the next question, how do you get these people involved in your movement to drive CM? Ultimately, it is these people that will own and manage a CM-based environment so you need their participation. It might be obvious by this point but I think someone should consider how they can incorporate the work of the script writers. How long will it take to build up expertise for a new solution anyway? How can one bridge between the old and new paradigms?

The pragmatic answer is to start with what got you there. Start with the scripts but figure out a way to cleanly plug them in to a CM management paradigm. Plan for the two styles of automation (procedural scripting vs CM). Big enterprises can’t throw out all the old and bring in the new in one shot. From political, project management, education, and technology points of view, it’s got to be staged.

To facilitate this pragmatic move towards full CM, script writers need:
  • A clean consistent interface. Make integration easy.
  • Modularity so new stuff can be swapped/plugged in later.
  • Familiar environment. It must be nice for shell scripters
  • Easy distribution. Make it easy for a shell scripter to hand off a tool for a CM user (or anybody else for that matter)
Having these capabilities drives the early collaboration that is critical to the success of later CM projects. From the shell scripter’s point of view, these capabilities put some sanity, convention and a bit of a framework around how scripting is done. 

I know this mismatch between the old shell script way and the new CM way all too well. I’ve had to tackle this problem in several large enterprises. After a while, a solution pattern emerged. 

Since I think this is an important problem that the DevOps community needs to address, I created a GitHub project to document the pattern and provide a reference implementation. The project is called rerun. It’s extremely simple but I think it drives home the point. I’m looking forward to the feedback and hearing from others who have found themselves in similar situations.


For more explanation of the ideas behind this, see the “Why rerun?” page.
Devops Chicago and Devops Camp

Devops Chicago and Devops Camp

Dev2ops / 

Martin J. Logan @martinjlogan is the founder of and the DevOps Chicago meetup group. He is also an Erlang hacker and co-author of Erlang and OTP in Action. In his spare time, he serves as Director of Merchandising, SEO, and Mobile Technology at Orbitz Worldwide. In a former life, I had the pleasure of working with Martin on what would now be called a Platform as a Service (PaaS) team at Orbitz.

> @mattokeefe: Martin, when did you first hear about “DevOps”?

> @martinjlogan: For me it was about 10 years ago. I was working at a place called Vail Systems for one of the Camp DevOps speakers, Hal Snyder. He implemented CFEngine back then and got the company to a state of rather high production environment automation. I subsequently left Vail and thought for sure that moving on to larger companies I would be witness to amazing automation when compared with what I had seen at little ol Vail with Hal. I was shocked to find out that this was definitely not the case. This was the genesis of my discovery of DevOps and formal Ops automation which at the time was not called anything of the sort.

> @mattokeefe: Hal is working with you again now at Orbitz. Did you help to attract him with the mission of implementing DevOps-style automation?

> @martinjlogan: Indeed I did. I brought Hal in to Orbitz for a presentation on CM automation about 3 years ago. Everyone was impressed with the talk but the organization at the time was not quite ready for what he was showing us. Well, since then Orbitz has come a long way. Hal impressed our head of operations, Lou Arthur, that first time and I of course kept his name fresh in peoples heads. Some time later Lou, Hal and I had lunch some and Hal elaborated some very exciting concepts in automation; I think it was a done deal after that and an offer went out.

> @mattokeefe: Suppose you were to make another hire… would you look for a developer seeking to learn more about Ops, or a sysadmin looking to learn more about development?

> @martinjlogan: Well, that is an interesting question indeed. Companies tend to invest more heavily on the development side looking at Ops as more of a cost center than a driver for returns. DevOps is working to change this mentality. Spending money on broadening Ops is in line with looking at Ops as a revenue generator and I think most places are a bit unbalanced in this respect. At the end of the day though, I am looking for both. We need an engineer that is a sysadmin, or a sysadmin that is an engineer, and we need this person to help us build our Ops as a service that is more and more sophisticated and conducive to the efficient release of software to our customers.

> @mattokeefe: Can DevOps work in a company with highly centralized Operations? ITIL?

> @martinjlogan: There is a lot of technology that underpins DevOps and Continuous Delivery particularly but in a lot of ways DevOps is about breaking down walls. In a large organization there are a lot of walls. Any given organization will have appetite or see value, and subsequently benefit from, breaking down some of those walls.

> @mattokeefe: In some Agile orgs, walls are broken down by co-locating teams with all disciplines seated in the same area. Is this a good idea with DevOps too, or is it enough to perhaps have a war room where you can sit down together when needed?

> @martinjlogan: I am a big believer that DevOps is an extension of Agile in many ways. It is really taking Agile to its logical conclusion. In Agile we say, done means tested. Taken to its logical conclusion done means in production (in production of course implies tested). Agile is also again about breaking down walls and fostering the communication and feedback loops that allow for empirical process control to actually happen. If I want to be really Agile then I want to know when things are going wrong in in production, I want the whole team to feel the pain and solve the problem together and learn together and implement controls and improvements that solve the problem moving forward. I want them to own that together – so yes, I think sitting together fosters such more than does the use of a war room here and there. That said, I think putting whole teams in windowless rooms that were once used for meetings is cruel and unusual.

> @mattokeefe: What are some of the tools that your teams are using today? Do you find that developers and sysadmins have different preferences for tools?

> @martinjlogan: We have quite a variety of tools here. We are certainly big users of Graphite on the monitoring front. We are Jenkins users as well. We are moving over to Git in many places throughout the company. One of the reasons being that it fits in quite nicely with many third party tools like Gerrit. There is definitely some difference in the tools Dev and Ops naturally gravitate towards. I think this is natural. For example Ops has been a proponent of Puppet while there is a strong dev contingent advocating glu.

> @mattokeefe: Which session are you looking forward to the most at Camp DevOps?

> @martinjlogan: That is honestly a tough question. There are quite a few big brains presenting. Jez Humble is amazing and his book Chris Read speak a while back and he really impressed me as well with the tremendous depth of hands on practical knowledge he has. John Willis is also fantastic, he is quite a personality and has done so much for DevOps, not to mention the Chef expertise. It is honestly a difficult one to call. I guess if I had to pick for me right now, Pascal-Louis Perez. He was the CTO at Wealthfront where he moved them onto Continuous Deployment. Certainly doing such a thing for a financial services company takes some serious ingenuity. Very excited to learn from him. Really though, I am looking forward to the whole thing.