Introduction and Background
Since starting to work with Puppet Labs’ Puppet at my current position, I have been constantly looking to better understand how to use puppet, and for ways to improve how my team uses puppet so we can do more with less. In the process of this, I do a lot of reading and try to stay active on the mailing list, IRC and the new ask.puppetlabs site, both to further my own understanding and to give back to the community some.
I recently came across a pair of blog posts that really caught my eye. The first one was “Simple Puppet Module Structure Redux” by R.I.Pienaar. This post describes a very nice pattern that is a blend of the package – config – service pattern and a modified anchor pattern. It does rely on Puppet 3′s databinding, so does not use the params.pp pattern yet does have the same sort of flexibility. After reading it, I really liked it. But I did realize that it was really only applicable to a certain type of puppet module. What I consider to be the building blocks of a node, or application level modules. For modules like the classic example of NTP, or Apache2 or Tomcat, this pattern is amazing. However, it doesn’t necessarily fit with higher level modules, that are pulling in the different building block modules and how those are bound together to create servers and clusters an infrastructure. I was giving this some thought, and trying to catch R.I. online and I came across another blog post that was referenced on the users mailing list.
This post was titled “Designing Puppet – Roles and Profiles” and written by Craig Dunn. This one is about using layers of abstraction to reduce the learning curve of an infrastructure, make it more self-documenting and reduce code duplication. After some thought, I decided to implement both of these patterns with my own puppet use. I have not yet done so, but have started reworking them. After some discussions on IRC, I have decided to document this process and give some examples on how these patterns can work together. I will not be sharing the actual code I use at work, but I will be creating some real world examples and documenting the general process.
For the purposes of this series of posts, I will be puppetizing the following infrastructure:
- 1 database cluster
- 3 app server clusters
- 1 web server cluster
- 1 ldap cluster
- 1 load balancer cluster
- 1 content server
Each web server and app server cluster consists of three nodes that are identical save for the hostname. The ldap has one master and one replica. They both serve reads but only the master handles writes. The load balancer cluster is two servers, one an active master and the other a hot standby. The database cluster consists of a master and a slave that is replicated to and can serve as a hot standby. The load balancer is in front, and balances ldap and web requests. Each web server node also serves as a balancer for the app server clusters. All nodes will be running the same OS, in order to simplify the blog post, and because that is how I do things at my job.
Current State of Affairs
Currently, at least in my environment, my puppet code is a mix of things from back before “I knew better”, prior best practice “flavor of the week” patterns, and code that lays somewhere in between. It is inconsistent, and not always easy to maintain or understand. This is further complicated by the fact that there have been several different people working on the codebase, each with their own style and idea of what good puppet code should look like.
Part two will focus on how the planning and design phases, of how to get an existing mess into something that will be easy to maintain and manage. Part three will be filled with examples – both in terms of how to lay things out on the file system to be able to take advantage of the autoloader and Puppet 3′s databindings (I’ll even throw in a way you can get databinding like functionality in puppet 2.7.x) and actual code examples for roles, profiles and the application/service level modules that form the foundation for it all.