Puppet

Puppet

by Nathan Tippy, Principal Software Engineer

May 2013

Introduction

Puppet is an open source cross-platform software package for declarative configuration management.  Puppet configuration files use the .pp extension and define the desired final state for each node.   Platform specific package management tools are used by Puppet to ensure compliance with the configuration files.

These declarative configuration files permit a single operations person to manage many hundreds of servers. These same files permit testers to perfectly duplicate production environments as needed.  Of course developers can write these declarative files as a way to define the installation process and document the subtle dependencies that often exist between applications.

Puppet labs produces both an enterprise and an open source edition of Puppet but this article will only be covering the open source edition.  The enterprise edition adds support and a few usability features but most of this functionality is duplicated by 3rd party modules for the open source edition.

The flexibility of Puppet allows it to be used in many configurations for many purposes.  Here are a few of the more common deployment strategies.

Puppet is a cross-platform tool designed to enable the management of mixed environments including flavors of *nix, OS X and even Windows.  In general, this mixed approach works well with some notable exceptions.  Windows support is somewhat crippled because it only supports a subset of the resources that can be managed easily on other platforms.  Another significant limitation is that a Windows machine can not be used as the Puppet master.  Overall, however, Puppet provides a very consistent experience regardless of what operating system is in use.

Puppet Work Flow

Puppet always follows the same repeatable work flow for applying changes, regardless of the manner of deployment.  This process may be kicked off by push events, the default polling schedule which is set to 30 minutes, or manually requested at the command line.

  1. Manifests(Modules) are written by you or your friends.

    GitHub and PuppetForge both have large collections of modules.  A simple search for a module similar to the needed one will likely save a lot of time before starting a new one.  Utilizing revision control for modules is highly advisable, as with any other software project.  It may be tempting to directly modify the module files on the puppet machine, but this defeats the purpose of using Puppet by making it impossible to repeat the deployment on other machines.

  2. Clients(Nodes) will gather facts from facter.

    When Puppet is installed it comes with ‘facter’: a command line utility for listing all the facts that Puppet can use for driving decisions in modules.  There are a long list of facts supported such as IP, FQDN(Fully Qualified Domain Name), CPU speed, memory or Operating System.

  3. Applicable manifests are compiled into a catalog.

    The catalog is built specifically for this node and its facts.  The catalog is a directed acyclic graph of changes that must be applied or confirmed in order.  There can frequently be legs within the graph that have no ordering requirements.  This is a result of the declarative nature of the *.pp files.  In those situations no assumptions relating to the order of changes applied should ever be made.

  4. Ensure compliance with the newly built catalog.

    The local Puppet agent will walk through the catalog and apply configuration changes as needed.  Puppet keeps track of what was done on previous runs and can quickly validate whether the current state has or has not changed.  This greatly helps to speed up the process because installation work is not needlessly repeated.

  5. Report generation

    The Puppet master will be notified if the client installation is configured to report changes.   The Puppet master can then be queried to determine the state of all the nodes.  TheForeman is a great 3rd party add-on to provide a nice web based GUI for this data.  It has many other useful features and can be used to simplify the installation of Puppet.

Starting and Testing New Nodes

While effecting new development with Puppet, it is sometimes necessary to kick off a run before the next pull.  Thankfully, Puppet provides some tools for doing exactly this.

 >  sudo puppet agent --test --onetime

Notice that this command requires admin rights.  This makes sense because Puppet is capable of installing or removing practically anything on a node.  When the puppet master is installed, access to module files is restricted to ensure the security of the cluster.  When new client nodes first talk to the agent, a signed certificate is required before access is granted to any data.  This is not a process that can or should be automated; it requires a human in the loop to confirm that access should be granted to the new client machine.

  1. Client’s first request
  1. > sudo puppet agent --test --onetime

    2.  Master lists the pending certificate requests

  1. > sudo puppet cert list

    3.  Administrator signs the certificate

  1. > sudo puppet cert sign <NodeName>

    4. Client retries request

  1. > sudo puppet agent --test --onetime --noop

An Example (installing Java 7 on Ubuntu)

Puppet starts with the /etc/puppet/manifests/site.pp file and from there it will import other files and make use of installed modules.  Nodes can be managed by node.pp files normally imported into the top of the site.pp.  Tools such as TheForeman can be used as an ENC (External Node Classifier) to greatly simplify this work with an easy to use front end.  For the example here, however, we do everything by hand to demonstrate how these files work.

Nodes are always defined by fully qualified domain names in Puppet.   FQDNs are so important that you must ensure your machine has one before running any example or even installing Puppet.

In this example /etc/puppet/manifests/node.pp file we define some node groups using some simple regular expressions.  Then we use one simple include statement to declare which of our classes should be applied to the group.  It is possible to use a single parameterized class with different parameters for each group.  However, that might make the node file harder to read.  Many of the decisions relating to how modules and nodes are organized come down to a matter of taste.  Nevertheless, the DRY (Don’t Repeat Yourself) principle should always be respected.

  1. node /^www\d+\.ociweb\.com$/ {
  2.             include legacyJDK
  3. }
  4.  
  5. # this will match qa<number>.ociweb.com
  6. node /^qa\d+\.ociweb\.com$/ {
  7.             include stableJRE
  8. }
  9.  
  10. # this will match dev<number>.ociweb.com
  11. node /^dev\d+\.ociweb\.com$/ {
  12.             include stableJDK
  13. }
  14.  
  15. # this will match the single node with the FQDN of experimental.ociweb.com
  16. node 'experimental.ociweb.com' {
  17.             include earlyAccessJDK
  18. }

Configuration declarations should be pushed down into the modules as much as possible for greater reuse.  In the example below, only Java versions supported by this mythical enterprise have been defined in the site.pp file.  This prevents duplication by defining the Java versions only once.

  1. import 'myNodes.pp'
  2.  
  3. class legacyJDK {
  4.     class{ 'java':
  5.         version => '1.7.0_17',
  6.         tarfile =>  $::architecture ? {
  7.             'amd64' => 'jdk-7u17-linux-x64.tar.gz',
  8.             default => 'jdk-7u17-linux-i586.tar.gz',
  9.         },
  10.         force   => false
  11.     }
  12. }
  13.  
  14. class stableJDK {
  15.     class{ 'java':
  16.         version => '1.7.0_21',
  17.         tarfile =>  $::architecture ? {
  18.             'amd64' => 'jdk-7u21-linux-x64.tar.gz',
  19.             default => 'jdk-7u21-linux-i586.tar.gz',
  20.         },
  21.         force   => false
  22.     }
  23. }
  24.  
  25. class stableJRE {
  26.     class{ 'java':
  27.         version => '1.7.0_21',
  28.         tarfile =>  $::architecture ? {
  29.             'amd64' => 'jre-7u21-linux-x64.tar.gz',
  30.             default => 'jre-7u21-linux-i586.tar.gz',
  31.         },
  32.         force   => false
  33.     }
  34. }
  35.  
  36. class earlyAccessJDK {
  37.     class{ 'java':
  38.         version => '1.8.0',
  39.         tarfile => $::architecture ? {
  40.             'amd64' => 'jdk-8-ea-bin-b79-linux-x64-28_feb_2013.tar.gz',
  41.             default => 'jdk-8-ea-bin-b79-linux-i586-28_feb_2013.tar.gz',
  42.         },
  43.         force   => true
  44.     }
  45. }

All modules are loaded from the init.pp file as their starting point.  Often modules include other files and templates.  In this example all the files for the module are found in /etc/puppet/modules/java.  It is required that modules are put in a folders matching their class name, in this case “java” .

There are many features not addressed here, such as templates, but this simple example demonstrates the most common ones.  The full example is available on github.

Note the heavy use of meta parameters to enforce the order of work to be completed.  It is easy to think that the work will be done top-down as it is written but that would be wrong.  These files are all declarative so any required dependencies must always be explicitly declared. Puppet labs has an exhaustive reference on line for further study.

  1. class java($version, $tarfile, $force=false) {
  2.     # Takes 3 parameters and the third one has a default of false.
  3.     # Variables in puppet can only be assigned once.
  4.     # Here we build some simple strings we will need later.
  5.  
  6.     # These are all the binaries provided by the JRE.
  7.     $jrebins = 'java,javaws,keytool,orbd,pack200,rmiregistry,servertool,tnameserv,unpack200'
  8.  
  9.     $jdk1bins = 'appletviewer,extcheck,idlj,jar,jarsigner,javac,javadoc'
  10.     $jdk2bins = 'javah,javap,jconsole,jdb,jhat,jinfo,jmap,jps,jrunscript'
  11.     $jdk3bins = 'jsadebugd,jstack,jstat,jstatd,native2ascii,policytool,rmic'
  12.     $jdk4bins = 'rmid,schemagen,serialver,wsgen,wsimport,xjc'
  13.  
  14.     # Puppet does not have a concat operator for strings however it does have
  15.     # interpolation when the " double quote is used.  Making use of the
  16.     # variables defined above a single large string is built.
  17.     # These are all the binaries provided by the JDK.
  18.     $jdkbins =  "${jdk1bins},${jdk2bins},${jdk3bins},${jdk4bins}"
  19.  
  20.     # If the string 'jre' or 'jdk' is found in the tar file name we set the
  21.     # appropriate values for $type and $bins
  22.     # The file copy operation from the master to this node is done only if its
  23.     # recognized to be a jre or jdk.  Further down in the exec for untar the
  24.     # subscribe metaparameter is used to continue the install ONLY if this
  25.     # file gets created:         subscribe => File["/tmp/${tarfile}"]
  26.     if jre in $tarfile {
  27.         $type = 'jre'
  28.         $bins = $jrebins
  29.  
  30.         file { "/tmp/${tarfile}":
  31.             ensure => file,
  32.             source => "puppet:///modules/java/${tarfile}",
  33.         }
  34.     } elsif jdk in $tarfile {
  35.         $type = 'jdk'
  36.         $bins = "${jrebins},${jdkbins}"
  37.  
  38.         file { "/tmp/${tarfile}":
  39.             ensure => file,
  40.             source => "puppet:///modules/java/${tarfile}",
  41.         }
  42.     } else {
  43.         alert('ensure the tar file name contains substring jre or jdk')
  44.         # File in temp folder is not created so the install stops.
  45.     }
  46.  
  47.     # Warn users that this was only intended for Debian platforms but
  48.     # the install will continue anyway
  49.     if $::osfamily != 'Debian' {
  50.         alert("This module only tested with Debian osfamily but ${::osfamily} was detected, use at your own risk.")
  51.     }
  52.  
  53.     # Ensure that the directory for jvm exists
  54.     # Require is used by the exec for untar below to ensure the right ordering.
  55.     file { '/usr/lib/jvm':
  56.         ensure => directory,
  57.         owner  => 'root',
  58.         group  => 'root',
  59.     }
  60.  
  61.     # The exec for untar uses the creates metaparameter to tell puppet not to
  62.     # bother running the command again if the creates file exists.
  63.     # When we need to change whats inside the tar we need to force it by
  64.     # ensuring the expected folder name destination is absent.
  65.     if $force == true {
  66.         file { "/usr/lib/jvm/${type}${version}" :
  67.             ensure => absent,
  68.             force  => true,
  69.             before => Exec["untar-java-${type}${version}"],
  70.         }
  71.     }
  72.  
  73.     # untar new Java distros into the right version named folder
  74.     # Will not run if the creates=> folder already exists
  75.     # Will not run if the require=> folder user/lib/jvn does not exist
  76.     # Will not run if the subscribe=> file has not been created
  77.     exec { "untar-java-${type}${version}":
  78.         command   => "/bin/tar -xvzf /tmp/${tarfile}",
  79.         cwd       => '/usr/lib/jvm',
  80.         user      => 'root',
  81.         creates   => "/usr/lib/jvm/${type}${version}",
  82.         require   => File['/usr/lib/jvm'],
  83.         subscribe => File["/tmp/${tarfile}"],
  84.     }
  85.  
  86.     # Splits a string on the the dot token and creates array versionarray
  87.     $versionarray = split($version, '[.]')
  88.     $jvmfolder = "/usr/lib/jvm/java-${versionarray[1]}-oracle"
  89.  
  90.     # Subscribes to the exec for untar so if executed
  91.     # it will then add a symlink to the version folder
  92.     # force must be used just in case a symlink already exists but
  93.     # it is pointing at some old location.
  94.     file { $jvmfolder:
  95.         ensure    => link,
  96.         force     => true,
  97.         target    => "/usr/lib/jvm/${type}${version}",
  98.         subscribe => Exec["untar-java-${type}${version}"],
  99.     }
  100.  
  101.     # Parse the string of binaries on the comma and produce an array
  102.     $binsarray = split($bins, '[,]')
  103.  
  104.     # This call to the define works like a macro.
  105.     # The $binsarray values are each mapped to $name causing multiple
  106.     # exec commands to get called based on what is in the array.
  107.     altinstall{ $binsarray:
  108.         jvmfolder => $jvmfolder
  109.     }
  110. }
  111.  
  112. # Using define to create 3 sets of execs
  113. # to update the alternatives for these bins
  114. define altinstall ($jvmfolder) {
  115.  
  116.     # Install this alternative only if the sim link is created
  117.     # the command its self requires double quotes but double quotes are also
  118.     # in use by puppet because the string is interpolated.  In order to make this
  119.     # work the quotes needed by the command are escaped with \"
  120.     exec { "alt-install-${name}":
  121.         command   => "/usr/sbin/update-alternatives --install \"/usr/bin/${name}\" \"${name}\" \"${jvmfolder}/bin/${name}\" 1",
  122.         subscribe => File[$jvmfolder],
  123.     }
  124.  
  125.     # Set this version as the active default if it is installed
  126.     exec { "alt-set-${name}":
  127.         command   => "/usr/sbin/update-alternatives --set \"${name}\" \"${jvmfolder}/bin/${name}\"",
  128.         subscribe => Exec["alt-install-${name}"]
  129.     }
  130. }

Due to Puppet’s flexibility and broad platform support it has become a very popular way to eliminate tedious and error-prone tasks.  Especially now that it is necessary to support an ever-growing number of machines that are often in the cloud.  Puppet helps leverage the knowledge from development, testing and operations in order to ensure consistent, repeatable deployments.

Puppet is an excellent tool for managing large clusters of machines.  This article has demonstrated only one possible way to make use of Puppet.  How do you plan to use it?

References