As our number of server need to be increased in order to improve the quality of our service, we need a way to manage our servers. One program we found to be an option is Puppet. This document explains what is puppet and how to use it.

2 What is Puppet?

Puppet is a configuration management tool. By creating a Puppetmaster server, it allows to manage multiple servers by assigning them as puppet to this central server.

All Puppet's configurations are stored on the Puppetmaster in manifests files. The language used in those files is object oriented. In those files are written the whole configuration of your servers. Packages, Files, Users, Services can be defined using the predefined types. Furthermore, you can define your own types to extend your possibilities.

This tool make possible to ensure a server is not only installed but is at a defined state. For example, you can check that a service is still running.

Puppet is written in Ruby and it is possible to use the Ruby language to add some functionalities.

3 Why Puppet?

The aim of puppet is to grant an overall control of numerous servers. It can make them homogenous and suppress the repetition of the same tasks.

When you have a ready configuration for a type of server, it takes very short time to deploy new servers.

See the chapter of known tools [known_tools] for a quick overview of all Puppet like tools.

4 Pros & Cons

4.1 Advantages:

A large community with mail list, irc channels is turning around Puppet

This ensures a good documentation and an active development.

Give viewable files which encourage the understanding and representation of a server configuration.

Provide an abstract management multiple servers.

4.2 Disadvantages:

Not designed for few server configurations.

Takes times to read all documentation and know the best practices to save time.

5 Architecture

The Puppetmaster is a centralized server which contains configurations specifications and files for its clients.

The clients, also called puppets, will pull regularly the state defined on the Puppetmaster to check if any changes have to be made to fit this state.

6 Installation

6.1 Server side

aptitude update

aptitude install puppetmaster

hostname puppet.switzernet.com

puppetmasterd –-mkusers --

vi /etc/hosts

Add a line like this in /etc/hosts:

ipaddress puppet.switzernet.com

To setup your vi with puppet syntax you can find the needed files here :

ftdetect

wget --no-check-certificate https://github.com/puppetlabs/puppet/raw/master/ext/vim/ftdetect/puppet.vim

syntax

wget --no-check-certificate https://github.com/puppetlabs/puppet/raw/master/ext/vim/syntax/puppet.vim

And follow the following how-to replacing the file for Kamailio by the ones fo puppet to set them up: [doc]

6.2 Client side

All you have to do, on client's side, is configuring dns and install puppet.

aptitude update

aptitude install puppet

hostname hostname

vi /etc/hosts

Add a line like this in /etc/hosts:

ipaddress client.domain.name

6.3 Subscribe a client

On your client, launch the following command:

puppetd --server puppet.switzernet.com --waitforcert 60 --test

Go on puppetmaster:

puppetca --list

It will show you the list of waiting client for certificates.

Sign the client to finish the subscription

puppetca -sign client.domain.name

7 Folders and files on Puppetmaster

In order to manage configurations it is necessary to keep all files needed in proper locations.

Here is the hierarchy on the Puppetmaster.

7.1 /etc/puppet

fileserver.conf

puppet.conf

This folder is the root of almost everything you need in puppet.

It also contains the conf files puppet.conf and fileserver.conf.

fileserver.conf is the configuration file to define the paths of modules, environments and file serving.

7.2 /etc/puppet/manifests

site.pp

nodes.pp

In this folders are the main definitions of configuration.

site.pp is the root of all classes. It is in this file that we have to call any other file.

nodes.pp describes for each node its specific configuration.

7.3 /etc/puppet/files

shared files

This folder contains the files available for clients.

Clients can reach those files using source parameter with puppet protocol :

file{"/myfile"

ensure => exists,

source => "puppet:///files/myfile"

}

7.4 /etc/puppet/modules

Modules needs to be stored in this folder

7.5 /etc/puppet/modules/<module>/manifest

init.pp

All manifests (.pp extention) which are part of the module must be in this folder.

The main file is init.pp

7.6 /etc/puppet/modules/<module>/files

shared files

This is like the folder /etc/puppet/files, but for files contained necessary to the module.

To get files in such case :

file{"/myfile"

ensure => exists,

source => "puppet:///mymodule/myfile"

}

7.7 /etc/puppet/modules/<module>/templates

tempates.erb

This folder contains some shared files that will be processed by facter before being copied on the right location on client.

8 Manifests syntax

Here is an overview of syntax used in manifest files (.pp).

It is object oriented designed. There are nodes, classes, types and defines. Those are called resources.

8.1 Nodes

A node defines a type of server. You can specify multiple servers under the same node.

To create a node (what we will do in nodes.pp) you can define it by multiple way.

You can make a type of node idenfied by a name. Be aware that default is a reserved name to say it is applied to all nodes.

node mynode { ... }

You can specify an ip.

node "192.168.1.1" { ... }

Or a domain name.

node "astrad.switzernet.com" { ... }

You cab cover multiple server using regular expression rules.

Since 0.25.0

node /^astrad[3-4]\.switzernet\.(com¦ch)$/ { ... }

8.2 Classes

A class is the first type of resource collection. Class can group resources to make it easier to manage.

Class act as a Singleton, it is only possible to init it once per node.

class myclass {

file{"/etc/example1": source => "/first/path"}

file{"/etc/example2": require => Package['firstpackage']}

file{"/etc/example3": require => Package['firstpackage']}

}

A class can inherits from an other class :

class mychildclass inherits myclass {

File{"/etc/example1": source => "/second/path"} # this redefine the source of the the file

File{"/etc/example2": require +> Package['secondpackage']} # this add another requirement

File{"/etc/example3": require => undef} # the reserve word undef remove all requirements

}

When you inherit from a class, you can modify its resources like in the previous example.

It is possible to use nested classes. To include a nested class, you have to use "::" to specify it.

class myclass {

class nested {

file { '/etc/passwd':

owner => 'root',

group => 'root',

mode => 644;

}

class anotherclass {

include myclass::nested

}

8.3 Defines

A define is the second type of resource collection. It allows you to create a group of resource that you can use multiple times defining various parameters.

Variables are written with a $ like in php.

You have to know that $title and $name are parameters always set in a define. Basically those variables are the same but you can set $title with a different name.

define copy_whole_dir ($source, $dest_dir, $user = "root") { # a default value is set to user

file{"$dest_dir/$name":

ensure => directory,

source => $source,

owner => $user

}

copy_whole_dir {

"example1": # this sets name and title

source => "/etc/tmp",

dest_dir => "/home/user";

"example2": # this define a second "copy_whole_dir" resource

source => "/etc/tmp",

dest_dir => "/var/lib",

user => "me"

}

8.4 Standard types

As you have seen it previously, types are defined that way :

<type> { <name1> :

param1 => values or [list_of_values],

param2 => values or [list_of_values],

…

paramN => values or [list_of_values];

<name2> :

param1 => values or [list_of_values],

param2 => values or [list_of_values],

…

paramN => values or [list_of_values];

…

<nameN> :

param1 => values or [list_of_values],

param2 => values or [list_of_values],

…

paramN => values or [list_of_values];

}

A list of value is defined that way:

[value1, value2, value3]

And when it is a list of resources:

Note the capital letter on the type of resource

[Resourcetype1['name1'], Resourcetype2['name2'], … ResourcetypeN['nameN']]

Here is a selection of few useful types to use :

9 Templates

A template is a text file in which some variable are defined. Those variables will be replaced when downloading.

There are few predefined variables called facts. Here is the list:

ipaddress ex. 192.168.1.17

hostname ex. astrad7

domain ex. switzernet.com

fqdn ex. astrad7.switzernet.com

operatingsystem ex. Debian

To check the availability of a fact on a client here is the command:

facter --puppet fact

To get the full list on a client, simply do :

facter

In manifests use the function template()

10 Execution order with metaparameters

Puppet will do what he can irrespectively to the order actions are described in the files.

To get more control on the process order, like making sure software has been installed before trying to use its functions,

Metaparameters are available in every type.

10.1before and require

Those metaparameters define what has to be done before and after a resource.

file{"/example ": ensure => directory}

file{"/example/path/myfile": }

file{"/exemple/path":

ensure => directory,

before => File["/exemple/path/myfile"]

require => File["/exemple"]

}

In this example we ensure the processing of folder creation in the proper order so we won't try to create a file in an inexistent folder.

10.2subscribe and notify

Those metaparemeters permit to do conditional processing. Like if this filed changed we want a service to restart.

file{"/example ":

ensure => file

notify => Exec["another_service restart"]

}

exec{"myservice restart ":

ensure => directory,

subscribe => File["/exemple"]

}

exec{"another_service restart":

ensure => directory,

}

10.3stage

Since 2.6.0

Some times, we have a group of resources to process before another group of resource. And defining all metaparameters one by one may be quite annoying and a big waste of time. Furthermore you have to test a dozen of time that you didn't missed any of the dozen of dependencies you have to create.

In order to make "groups" of instructions, stage is a metaparameter that can be used.

This metaparameter is only available on class

11 Generate dependencies graph

Sometimes you might experience an issue of dependencies or you want to have a look on dependencies.

Puppet includes a auto generation of graphs

This is an example of generated graph.

To generate this kind of graph, you have to enable the graph generation.

On the client launch puppet with --graph true.

Here is an example.

puppet --server puppet.switzernet.com --test --graph true

It will generate graphs file (.dot) in the default graph folder : /var/lib/puppet/state/graphs/

You can then upload those graphs on your local computer with scp :

scp root@astrad6.switzernet.com:/var/lib/puppet/state/graphs/*

To be able to read those files, you will need a software.

Here is graphviz, a tool that will generate images (gif, jpg, png, …) from graph files :

http://www.graphviz.org/Download_windows.php

When graphviz is installed, launch Gvedit.exe. Open relationship.dot and click on the run button

Then, select the type of image you want to generate and click Ok.

This will open a window in graphviz with the graph generated. Plus, it will add a file at the "Output File Name" destination. By default, it will be at the same location as the .dot file itself.

12 Puppet version

12.1Language Feature by Release

Feature	0.23.1	0.24.6	0.24.7	0.25.0	2.6.0
Plusignment operator (+>)	X	X	X	X	X
Multiple Resource relationships		X	X	X	X
Class Inheritance Overrides		X	X	X	X
Appending to Variables (+=)		X	X	X	X
Class names starting with 0-9		X	X	X	X
Multi-line C-style comments			X	X	X
Node regular expressions				X	X
Expressions in Variables				X	X
RegExes in conditionals				X	X
Elsif in conditionals					X
Chaining Resources					X
Hashes					X
Parameterised Class					X
Run Stages					X
The “in” syntax					X

12.2Debian

Unfortunately, on Debian depositories, only puppet 0.24.5 is available. This makes unavailable some functionality.

In order to install a newer version to test it here is the procedure.

vi /etc/apt/source.list

Add this line

deb http://www.backports.org/debian lenny-backports main contrib non-free

aptitude update && aptitude install -t lenny-backports puppetmaster

13 Errors and solutions

13.1Could not retrieve catalog: Could not parse for environment production: Could not match

In case of a parse error like this one:

err: Could not retrieve catalog: Could not parse for environment production: Could not match '$http_user' at /etc/puppet/manifests/nodes.pp:3

Your file might be in windows format because you uploaded it via ftp.

In such case, you can edit the file with vi and use the following command:

:set fileformat=unix

13.2Failed to retrieve current state of resource: No specified source was found from <source>

In case of an error like this :

err: //Node[default]/astrad/asterisk/File[/usr/share/asterisk/sounds/FR-invalid-Account.gsm]: Failed to retrieve current state of resource: No specified source was found from puppet:///asterisk/sound/FR-invalid-Account.gsm

You have a problem with your files rights on your puppetmaster.

Check that your files are –rw-r--r-- (644) and your folders drwxr-xr-x (755)

13.3Failed to generate additional resources during transaction: Certificates were not trusted: hostname was not match with the server certificate

err: /File[/var/lib/puppet/lib]: Failed to generate additional resources during transaction: Certificates were not trusted: hostname was not match with the server certificate

Here you have a problem in the certificates because you are trying to reach the server using another domain name that it was used when certificates were signed.

Makes sure the parameter --server is correct.

If it is, makes sure your server sign with the right domain name.

To ensure this, launch this command on puppetmaster :

puppetmaster --certname puppet.switzernet.com

13.4Found dependency cycles in the following relationships

err: Could not apply complete catalog: Found dependency cycles in the following relationships: File[/etc/asterisk] => Exec[asterisk restart], File[/etc/asterisk/asterisk.conf] => Exec[asterisk restart], File[/etc/asterisk/manager.conf] => Exec[asterisk restart], File[/etc/asterisk/ext-radius.conf] => Exec[asterisk restart], File[/etc/asterisk/default-confs] => Exec[Backup default config files], File[/etc/asterisk] => File[/etc/asterisk/ext-radius.conf], File[/etc/asterisk] => File[/etc/asterisk/manager.conf], Exec[asterisk restart] => Exec[Restart Alert@], File[/etc/asterisk] => File[/etc/asterisk/asterisk.conf], Exec[Backup default config files] => File[/etc/asterisk], File[/etc/asterisk] => File[/etc/asterisk/default-confs]

In such case, you have a cycle in your dependencies. To find it, you better have to generate a graph.

This will help you to save a lot of time.

To see how to generate a graph see [graph]

Then suppress the cycle by editing require, subscribe, before or notify statements.

Here is an example of dependency cycle found.

It is important to know that a dependency seems to be created between a file resource and another parent file resource.

For example : file{"/home/user/myfolder"} will depend from file{"/home/user"}

14 Known tools

All well known configuration management tools are the followings. All offer a medium of configuration management.

14.1capistrano

written in Ruby
Stable release 2.5.15 / 14 Feb. 2010
Doesn't take care of dependencies when installing packages.

This is a deployment tool and not a management tool.

Example of file:

Given /^a an app$/ do

# Create the git repo

FileUtils.mkdir_p @repo_dir

Dir.chdir(@repo_dir) do

system "git --bare init"

end

# Create and capify the dummy app, and push it to the local repo

FileUtils.mkdir_p @app_dir

Dir.chdir(@app_dir) do

[

%Q{git init},

%Q{mkdir config},

%Q{capify .},

%Q{git add .},

%Q{git commit -m "first commit"},

%Q{git remote add origin file://#{@repo_dir}},

%Q{git push origin master}

].each do |command|

system command

end

# Write a custom deploy file to the app, using an ERB template

deploy_variables = {

:deploy_to => File.join(@test_files_dir, "deployed"),

:repository => @repo_dir,

:git_executable => `which git`.strip,

:logged_in_user => Etc.getlogin

}

template_path = File.expand_path(File.join(__FILE__, "..", "..", "templates", "deploy.erb"))

compiled_template = ERB.new(File.read(template_path)).result(binding)

File.open(File.join(@app_dir, "config", "deploy.rb"), 'w') {|f|

f.write compiled_template

}

end

When /^I deploy$/ do

Dir.chdir(@app_dir) do

system "cap deploy:setup"

system "cap deploy"

end

Then /^the PEOPLE_LIKE_YOU file should be written to shared$/ do

File.exists?(File.join(@test_files_dir, "deployed", "shared", "PEOPLE_LIKE_YOU")).should be_true

end

14.2bcfg2

written in python
Stable release 1.1.1 / November 15, 2010
Hard to read conf in XML file content.

Exemple of file :

</BoundPackage>

</Bundle>

14.3cfengine

Written in Ruby
Stable release 3.0.5 / June 07, 2010
Limited at a complexity level

body common control

{

bundlesequence => { "central" };

}

############################################

bundle agent central

{

vars:

"policy_server" string => "myhost.domain.tld";

"mypackages" slist => {

"nagios"

"gcc",

"apache2",

"php5"

};

files:

# Password management can be very simple if all hosts are identical

"/etc/passwd"

comment => "Distribute a password file",

perms => mog("644","root","root"),

copy_from => secure_cp("/home/mark/LapTop/words/RoadAhead","$(policy_server)");

packages:

"$(mypackages)"

package_policy => "add",

package_method => generic;

# Add more promises below ...

}

#########################################################

# Server config

#########################################################

body server control

{

allowconnects => { "127.0.0.1" , "::1", "10.20.30" };

allowallconnects => { "127.0.0.1" , "::1", "10.20.30" };

trustkeysfrom => { "127.0.0.1" , "::1", "10.20.30" };

# allowusers

}

#########################################################

bundle server access_rules()

{

access:

# myhost.domain.tld makes this file available to 10.20.30*

myhost_domain_tld::

"/etc/passwd"

admit => { "127.0.0.1", "10.20.30" };

}

14.4puppet

written in Ruby
Stable release 2.6.4 / December 1st 2010
Use more resources than CFEngine

class mysql-server {

package { "mysql-server": ensure => installed }

package { "mysql-client": ensure => installed }

service { "mysql":

ensure => running,

enable => true,

name => "mysql",

require => Package["mysql-server"],

}

exec { "Set MySQL root password":

subscribe => Package["mysql-server"],

refreshonly => true,

unless => "mysqladmin -uroot -p$mysql_root_pass status",

path => "/bin:/usr/bin",

command => "mysqladmin -uroot password $mysql_root_pass",

require => Service["mysql"],

}

14.5chef

written inRuby
Stable release 0.9.8 / August 5 2010
Very new product not developed enough.

Little community.

15 References

15.1Installation

http://www.ctrip.ufl.edu/install-puppetmaster-puppets-debian-lenny

http://www.debian-administration.org/articles/526

http://projects.puppetlabs.com/projects/puppet/wiki/Simplest_Puppet_Install_Pattern

http://www.deimos.fr/blocnotesinfo/index.php?title=Puppet_:_Solution_de_gestion_de_fichier_de_configuration

http://docs.puppetlabs.com/guides/configuring.html

http://www.devco.net/pubwiki/Puppet/GettingStarted/1

http://bitfieldconsulting.com/puppet-and-mysql-create-databases-and-users

15.10 CPAN

http://cpan.yimg.com/

http://stackoverflow.com/questions/3462058/how-do-i-automate-cpan-configuration/3462086#3462086

http://serverfault.com/questions/168644/cpan-installer-for-puppet-cant-find-relationship-target

http://www.mailinglistarchive.com/html/puppet-users@googlegroups.com/2010-08/msg00144.html

http://www.mail-archive.com/module-build@perl.org/msg01288.html

http://www.octopuce.fr/Puppet-Administration-systeme-centralisee

http://doc.ubuntu-fr.org/puppet

http://webcache.googleusercontent.com/search?q=cache:dhTod1c7im0J:www.lab42.it/presentations/puppetmodules/puppetmodules.html+puppet+%22line+%3D%3E%22+ensure+present&cd=8&hl=fr&ct=clnk&gl=ch&client=firefox-a

http://www.lab42.it/presentations/puppetmodules/puppetmodules.html

http://projects.puppetlabs.com/projects/1/wiki/Documentation_Start

http://docs.puppetlabs.com/guides/best_practices.html

http://www.linuxforu.com/how-to/puppet-show-automating-unix-administration/

http://blog.manjusri.org/2009/11/19/puppet-for-absolute-beginners/

http://narrabilis.com/book/export/s5/23

https://wiki.koumbit.net/Puppet#Introduction_.2BAOA_puppet

15.26 Files

SCaLE Puppet Introduction - [Dan_Bode.pfd]

Gestion d'infrastructure avec Puppet- [Puppet_RAISIN_final-2.pdf]

* * *