Tim Sharpe - Puppet2020-09-15T23:39:41+00:00https://bombasticmonkey.com/Tim Sharpetim@sharpe.id.auCopyright (c) 2020 Tim Sharpepuppet-lint 1.0.02014-08-19T00:00:00+00:00https://bombasticmonkey.com/2014/08/19/puppet-lint-1.0.0/<p>It’s been a long time coming but I’m happy to announce the release of
puppet-lint 1.0.0!</p>
<p>Along with a bunch of bugfixes and a rewrite of most of the code, there’s some
(hopefully) exciting new features in this release.</p>
<h2 id="automatic-fixing-of-errors">Automatic fixing of errors</h2>
<p>Previewed in the 0.4.0 pre-release, simple problems can now be automatically
fixed by puppet-lint (some problems require complex refactoring and so
puppet-lint won’t attempt anything on your behalf).</p>
<p>In 1.0.0, problems detected by the following checks can be automatically fixed
by running puppet-lint with <code class="highlighter-rouge">--fix</code>:</p>
<ul>
<li><code class="highlighter-rouge">slash_comments</code></li>
<li><code class="highlighter-rouge">star_comments</code></li>
<li><code class="highlighter-rouge">unquoted_node_name</code></li>
<li><code class="highlighter-rouge">unquoted_resource_title</code></li>
<li><code class="highlighter-rouge">unquoted_file_mode</code></li>
<li><code class="highlighter-rouge">file_mode</code></li>
<li><code class="highlighter-rouge">ensure_not_symlink_target</code></li>
<li><code class="highlighter-rouge">double_quoted_strings</code></li>
<li><code class="highlighter-rouge">only_variable_string</code></li>
<li><code class="highlighter-rouge">variables_not_enclosed</code></li>
<li><code class="highlighter-rouge">quoted_booleans</code></li>
<li><code class="highlighter-rouge">hard_tabs</code></li>
<li><code class="highlighter-rouge">trailing_whitespace</code></li>
<li><code class="highlighter-rouge">arrow_alignment</code></li>
</ul>
<p>Complimentary to <code class="highlighter-rouge">--fix</code>, puppet-lint now has a <code class="highlighter-rouge">--only-checks</code> parameter that
you can pass a comma seperated list of checks that you want to run. This works
great when doing an initial pass over your code to fix problems. Let’s say
that you just want to realign all your <code class="highlighter-rouge">=></code>s, run <code class="highlighter-rouge">puppet-lint --fix
--only-checks arrow_alignment modules/</code>.</p>
<h2 id="plugin-system">Plugin system</h2>
<p>From it’s inception, puppet-lint has enforced the official <a href="https://docs.puppetlabs.com/guides/style_guide.html">style
guide</a> and I’ve had to
turn down a lot of good ideas and contributions because they’re not a part of
that guide. To that end, I’ve added a plugin system in 1.0.0 so that we can
add and distribute custom checks outside of the main code base.</p>
<p>My hope is that together as a community we can come up with some new checks and
influence new versions of the style guide. I’ve written
a <a href="http://puppet-lint.com/developer/tutorial/">tutorial</a> that steps through
how to write a basic check. If you’ve got a good idea for a check but aren’t
up to writing it yourself, create an issue in the puppet-lint repo with the
<a href="https://github.com/rodjek/puppet-lint/labels/new%20check">“new check” label</a>
and maybe someone else will be able to.</p>
<p>You can find the currently tiny list of <a href="http://puppet-lint.com/plugins/">community
plugins here</a>. They’re distributed as Ruby
Gems so you can easily install them however you’re currently managing
puppet-lint (<code class="highlighter-rouge">gem</code>, <code class="highlighter-rouge">bundler</code>, etc).</p>
<h2 id="control-comments">Control comments</h2>
<p>A much requested feature, you can now disable tests via special comments in
your code. Read more about it
<a href="http://puppet-lint.com/controlcomments/">here</a>.</p>
<h2 id="new-checks">New checks</h2>
<p>A couple of new checks have been added to core puppet-lint in 1.0.0.</p>
<ul>
<li><code class="highlighter-rouge">unquoted_node_name</code> - Checks for unquoted node names</li>
<li><code class="highlighter-rouge">puppet_url_without_modules</code> - Checks for fileserver URLs without the
modules mountpoint.</li>
</ul>
<h2 id="whats-gone">What’s gone</h2>
<p>The <code class="highlighter-rouge">class_parameter_defaults</code> check has been removed. This check was based on
a misreading of the style guide.</p>
<p>For a more complete list of changes, I’d recommend reading <a href="http://puppet-lint.com/changelog/">the
changelog.</a></p>
rspec-puppet 1.0.02013-12-05T00:00:00+00:00https://bombasticmonkey.com/2013/12/05/rspec-puppet-1.0.0/<p>It’s been a while, but finally there’s a new release of rspec-puppet.
Originally slated to be 0.2.0 but promoted to 1.0.0 due to a couple of
backwards incompatible changes.</p>
<h2 id="what-might-cause-you-problems">What might cause you problems</h2>
<h3 id="create_resource-matcher">create_resource matcher</h3>
<p>This deprecated matcher has been removed entirely now. If you still have code
using it, you should switch to using the generic <code class="highlighter-rouge">contain_<resource></code> matcher
instead.</p>
<h3 id="include_class-matcher">include_class matcher</h3>
<p>This matcher has now been deprecated and will be removed in the next major
release. The reason this is on it’s way out is that it doesn’t support
parameterised classes and this has caused a lot of confusion for many users.</p>
<p>If you use <code class="highlighter-rouge">include_class</code> anywhere, you’ll see the following depreciation
notice.</p>
<figure class="highlight"><pre><code class="language-text" data-lang="text">DEPRECATION WARNING: you are using deprecated behaviour that will
be removed from a future version of RSpec.
* include_class is deprecated.
* please use contain_class instead.</code></pre></figure>
<p>So, change:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
</pre></td><td class="code"><pre><span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">include_class</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">)</span> <span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>To:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
</pre></td><td class="code"><pre><span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">contain_class</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">)</span> <span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<h3 id="changes-to-how-parameters-are-matched">Changes to how parameters are matched</h3>
<p>In previous versions of rspec-puppet, parameter values were matched extremely
naively, where all values were flattened down to a string before comparison.
This means that <code class="highlighter-rouge">['b', 'ba']</code> would have been equal to <code class="highlighter-rouge">['bb', 'a']</code> for
example. As of 1.0.0, this behaviour has changed and we now compare data
structures like arrays and hashes correctly, so you may have to adjust your
tests accordingly.</p>
<h2 id="whats-new">What’s new</h2>
<h3 id="hiera-support">hiera support</h3>
<p>Set the path to your <code class="highlighter-rouge">hiera.yaml</code> file in your <code class="highlighter-rouge">RSpec.configure</code> block and
you’re good to go.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="code"><pre><span class="no">RSpec</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">c</span><span class="o">|</span>
<span class="c1"># snip</span>
<span class="n">c</span><span class="p">.</span><span class="nf">hiera_config</span> <span class="o">=</span> <span class="s1">'/path/to/your/hiera.yaml'</span>
<span class="k">end</span>
</pre></td></tr></tbody></table></code></pre></figure>
<h3 id="compile-matcher">compile matcher</h3>
<p>This new matcher should the be first thing in any rspec-puppet test suite. It
checks that the catalogue will compile correctly and that there are no
dependency cycles in the generated graph.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
</pre></td><td class="code"><pre><span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">compile</span> <span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>This matcher also has a chain method to enable checking that all dependencies
in the catalogue have been met - <code class="highlighter-rouge">with_all_deps</code>.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
</pre></td><td class="code"><pre><span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">compile</span><span class="p">.</span><span class="nf">with_all_deps</span> <span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>While at first glance, it might seem that this shouldn’t be optional however
there are cases where you might not want to test this (if, for example, you are
testing a module that notifies a resource in a different module).</p>
<h3 id="relationship-tests">relationship tests</h3>
<p>Some new additions to the <code class="highlighter-rouge">contain_<resource></code> matcher are the resource
relationship tests.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="code"><pre><span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">contain_file</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">).</span><span class="nf">that_requires</span><span class="p">(</span><span class="s1">'File[bar]'</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">contain_file</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">).</span><span class="nf">that_comes_before</span><span class="p">(</span><span class="s1">'File[bar]'</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">contain_file</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">).</span><span class="nf">that_notifies</span><span class="p">(</span><span class="s1">'Service[bar]'</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">contain_service</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">).</span><span class="nf">that_subscribes_to</span><span class="p">(</span><span class="s1">'File[bar]'</span><span class="p">)</span> <span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Regardless of how you define your relationships, either using the
metaparameters (<code class="highlighter-rouge">require</code>, <code class="highlighter-rouge">before</code>, <code class="highlighter-rouge">notify</code> and <code class="highlighter-rouge">subscribe</code>) or the chaining
arrows (<code class="highlighter-rouge">-></code>, <code class="highlighter-rouge"><-</code>, <code class="highlighter-rouge">~></code> and <code class="highlighter-rouge"><~</code>) these tests will work.</p>
<p>Testing the reverse of the relationship described in your Puppet code will also
work with these new methods. Take the following manifest for example:</p>
<figure class="highlight"><pre><code class="language-puppet" data-lang="puppet"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="code"><pre><span class="n">notify</span> <span class="p">{</span> <span class="s1">'foo'</span><span class="p">:</span> <span class="p">}</span>
<span class="n">notify</span> <span class="p">{</span> <span class="s1">'bar'</span><span class="p">:</span>
<span class="kp">before</span> <span class="p">=></span> <span class="nc">Notify</span><span class="p">[</span><span class="s1">'foo'</span><span class="p">],</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Both of the following tests will work:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
</pre></td><td class="code"><pre><span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">contain_notify</span><span class="p">(</span><span class="s1">'bar'</span><span class="p">).</span><span class="nf">that_comes_before</span><span class="p">(</span><span class="s1">'Notify[foo]'</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">contain_notify</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">).</span><span class="nf">that_requires</span><span class="p">(</span><span class="s1">'Notify[bar]'</span><span class="p">)</span> <span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<h3 id="only_with-tests">only_with tests</h3>
<p>Also new to the <code class="highlighter-rouge">contain_<resource></code> matcher are the <code class="highlighter-rouge">only_with</code> tests. Unlike
the <code class="highlighter-rouge">with</code> tests which only test that the specified parameters have been
defined, <code class="highlighter-rouge">only_with</code> tests that these are the <em>only</em> parameters passed to
a resource.</p>
<p>Like the <code class="highlighter-rouge">with</code> tests, you can specify a single parameter with the
<code class="highlighter-rouge">only_with_<parameter></code> method:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
</pre></td><td class="code"><pre><span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">contain_service</span><span class="p">(</span><span class="s1">'ntp'</span><span class="p">).</span><span class="nf">only_with_ensure</span><span class="p">(</span><span class="s1">'running'</span><span class="p">)</span> <span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Or, you can pass it a hash of parameters and values:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="code"><pre><span class="n">it</span> <span class="k">do</span>
<span class="n">should</span> <span class="n">contain_service</span><span class="p">(</span><span class="s1">'ntp'</span><span class="p">).</span><span class="nf">only_with</span><span class="p">(</span>
<span class="s1">'ensure'</span> <span class="o">=></span> <span class="s1">'running'</span><span class="p">,</span>
<span class="s1">'enable'</span> <span class="o">=></span> <span class="kp">true</span><span class="p">,</span>
<span class="p">)</span>
<span class="k">end</span>
</pre></td></tr></tbody></table></code></pre></figure>
<h3 id="resource-counting-matchers">resource counting matchers</h3>
<p>The last new matchers for this release are <code class="highlighter-rouge">have_resource_count</code>,
<code class="highlighter-rouge">have_class_count</code> and the generic <code class="highlighter-rouge">have_<resource>_resource_count</code>. As you
can guess, these matchers:</p>
<p>Count the total number of resources in the catalogue</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
</pre></td><td class="code"><pre><span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">have_resource_count</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Count the number of classes in the catalogue</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
</pre></td><td class="code"><pre><span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">have_class_count</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Count the number of resources of a particular type in the catalogue</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
</pre></td><td class="code"><pre><span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">have_exec_resource_count</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<hr />
<p>As always, if you find any bugs or have any suggestions for new functionality,
please create an issue <a href="https://github.com/rodjek/rspec-puppet/issues">here</a>.</p>
Fix Simple Problems With puppet-lint2013-01-28T00:00:00+00:00https://bombasticmonkey.com/2013/01/28/fix-simple-problems-with-puppet-lint/<p>Since the first release of puppet-lint, one of the top two feature requests has
been the ability to automatically fix trivial style issues (detecting problems
is all well and good but no one wants to manually fix the eleventy thousand
problems that have accumulated over the years).</p>
<p>To that end, I have just shipped a beta release of puppet-lint (0.4.0.pre1)
which includes the experimental fixing support! I highly encourage everyone to
give it a try and report any issues they find.</p>
<p>Currently, puppet-lint supports fixing a limited subset of detectable problems.
There will probably be more added to this list over time however some problems
will always require a human to decide how to proceed.</p>
<p>At this time, puppet-lint do the following for you:</p>
<ul>
<li>Converting <code class="highlighter-rouge">//</code> comments into <code class="highlighter-rouge">#</code> comments.</li>
<li>Quoting unquoted resource titles.</li>
<li>Quoting unquoted file mode strings.</li>
<li>Converting 3 digit octal file modes into 4 digit modes.</li>
<li>Converting double quoted strings without variables into single quoted
strings.</li>
<li>Converting double quoted strings that only contain a variable into an
unquoted variable.</li>
<li>Enclosing variables in double quoted strings that haven’t been enclosed in
braces.</li>
<li>Unquoting quoted boolean values.</li>
<li>Converting hard tabs into 2 space soft tabs.</li>
<li>Removing trailing whitespace.</li>
<li>Fixing arrow (<code class="highlighter-rouge">=></code>) alignment in resources and hashes.</li>
</ul>
<h1 id="caveat-emptor">Caveat Emptor</h1>
<p><strong>Running puppet-lint with fix mode enabled is a potentially destructive
action.</strong></p>
<p>I’m going to assume that your manifests are stored in some sort of version
control and that you’re comfortable discarding the changes puppet-lint makes if
you don’t like them.</p>
<h1 id="trying-it-out">Trying it out</h1>
<p>First of all install the new version, either with RubyGems:</p>
<figure class="highlight"><pre><code class="language-console" data-lang="console"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
</pre></td><td class="code"><pre><span class="go">gem install --pre puppet-lint -v 0.4.0.pre1</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>or Bundler:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
</pre></td><td class="code"><pre><span class="n">gem</span> <span class="s1">'puppet-lint'</span><span class="p">,</span> <span class="s1">'0.4.0.pre1'</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Then just call puppet-lint with the <code class="highlighter-rouge">-f</code> or <code class="highlighter-rouge">--fix</code> option</p>
<figure class="highlight"><pre><code class="language-console" data-lang="console"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="code"><pre><span class="gp">$</span><span class="w"> </span>puppet-lint <span class="nt">-f</span> test.pp
<span class="go">FIXED: double quoted string containing no variables on line 1
FIXED: string containing only a variable on line 3
</span><span class="gp">FIXED: indentation of =></span><span class="w"> </span>is not properly aligned on line 5
<span class="go">FIXED: unquoted file mode on line 5
FIXED: mode should be represented as a 4 digit octal value or symbolic mode on line 5</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>And the diff of the changes to my test file is</p>
<figure class="highlight"><pre><code class="language-diff" data-lang="diff"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
</pre></td><td class="code"><pre><span class="gh">diff --git a/test.pp b/test.pp
index 7cd1e93..4d455cf 100644
</span><span class="gd">--- a/test.pp
</span><span class="gi">+++ b/test.pp
</span><span class="p">@@ -1,6 +1,6 @@</span>
<span class="gd">-$foo = "/tmp/foo"
</span><span class="gi">+$foo = '/tmp/foo'
</span>
-file { "$foo":
<span class="gi">+file { $foo:
</span> ensure => present,
<span class="gd">- mode => 444,
</span><span class="gi">+ mode => '0444',
</span> }
</pre></td></tr></tbody></table></code></pre></figure>
<p>So, please try it out and <a href="https://github.com/rodjek/puppet-lint/issues/">create an issue on the
repository</a> if you run into any
problems!</p>
From Quick Hack To Serious Project2012-07-11T00:00:00+00:00https://bombasticmonkey.com/2012/07/11/from-quick-hack-to-serious-project/<p>You might have noticed that there hasn’t been a puppet-lint release in quite
some time and that I’ve become a bit lax when it comes to keeping up with bugs
and pull requests.</p>
<p>Puppet-lint originally started as something to fill in some time while
10,000 metres above the Tasman Sea and like all good hack projects, I kept
adding more and more code onto it to get each release out the door as quickly
as possible. This lead to a number of questionable design decisions, the
biggest of which was using Puppet’s lexer to tokenise the manifests that are
being analysed.</p>
<p>Don’t get me wrong, the Puppet lexer is fantastic and I doubt I can
write a better one, but as it’s a part of a separate active project, developing
against such a moving target results in constantly adding new corner cases with
every Puppet release. This has resulted in a messy code base that I haven’t
really desired to touch, to the detriment of the project.</p>
<h2 id="so-what-now">So what now?</h2>
<p>If you’ve been following
<a href="https://github.com/rodjek/puppet-lint/">the project on GitHub</a>, you might have
noticed a flurry of recent activity in the
<a href="https://github.com/rodjek/puppet-lint/compare/master...dust_bunny">dust_bunny branch</a>.
Over the last week, I made the call to rip out the Puppet lexer and replace it
with a simple one of my own. Along with removing the constantly moving target,
it has had a couple of other major benefits:</p>
<ol>
<li>Faster start up time, as we’re no longer loading Puppet.</li>
<li>A custom lexer tailored to our needs means we can now access additional data
that was previously discarded (like comments).</li>
</ol>
<p>This has also given me an excuse to start a much needed cleanup of the code to
ensure that it is consistent and (hopefully) well documented. Anyone who’s
tried to add a new check will know what I mean.</p>
<h2 id="great-why-are-you-telling-me">Great, why are you telling me?</h2>
<p>While everything I’ve described doesn’t sound very interesting from a features
point of view, it does represent a massive change to the internals of
puppet-lint and there’s a very real possibility of introducing regressions.</p>
<p>So, what I need you to do is install the pre-release version of 0.2.0</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gem install puppet-lint -v 0.2.0.pre1 --pre
</code></pre></div></div>
<p>And then run it over your manifests. If you run into any bugs, false positives
or negatives etc, please report them on the <a href="https://github.com/rodjek/puppet-lint/issues">project issue
tracker</a> and tag them with the
dust_bunny label.</p>
<p>Once any issues that have arisen as a result of these changes have been
resolved, I’ll go back through the old issues and get them fixed up before
pushing out a proper 0.2.0 release with all the features you’ve been waiting
for.</p>
<h2 id="tldr">TL;DR</h2>
<p>Lots of internal changes happened in puppet-lint recently and I need you to
help test it out and make sure no regressions have snuck in.</p>
<p>Install it, run it, report it. Ta!</p>
Automatically Test Your Puppet Modules With rspec-puppet, puppet-lint And Travis CI2012-03-02T00:00:00+00:00https://bombasticmonkey.com/2012/03/02/automatically-test-your-puppet-modules-with-travis-ci/<p>I’m going to assume you’ve got a Puppet module already on GitHub. To save
messing around with bundler on your local machine, I recommend installing
<code class="highlighter-rouge">puppet-lint</code> and <code class="highlighter-rouge">rspec-puppet</code> as system gems while you’re getting this all
set up.</p>
<figure class="highlight"><pre><code class="language-console" data-lang="console"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
</pre></td><td class="code"><pre><span class="gp">$</span><span class="w"> </span>gem <span class="nb">install </span>puppet-lint
<span class="gp">$</span><span class="w"> </span>gem <span class="nb">install </span>rspec-puppet
</pre></td></tr></tbody></table></code></pre></figure>
<h2 id="rspec-puppet">rspec-puppet</h2>
<p>First of all, let’s create a directory structure for your spec files</p>
<figure class="highlight"><pre><code class="language-console" data-lang="console"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="code"><pre><span class="gp">$</span><span class="w"> </span><span class="nb">mkdir</span> <span class="nt">-p</span> spec/classes spec/defines spec/fixtures/manifests
<span class="gp">$</span><span class="w"> </span><span class="nb">mkdir</span> <span class="nt">-p</span> spec/fixtures/modules/<your module name>
<span class="gp">$</span><span class="w"> </span><span class="nb">cd </span>spec/fixtures/modules/<your module name>
<span class="gp">$</span><span class="w"> </span><span class="nb">touch </span>spec/fixtures/manifests/init.pp
<span class="gp">$</span><span class="w"> </span><span class="k">for </span>i <span class="k">in </span>files lib manifests templates<span class="p">;</span> <span class="k">do </span><span class="nb">ln</span> <span class="nt">-s</span> ../../../../<span class="nv">$i</span> <span class="nv">$i</span><span class="p">;</span> <span class="k">done</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>If you’re wondering about that last line, we symlink the contents of the module
into <code class="highlighter-rouge">spec/fixtures/modules/<your module name></code> so that we can trick Puppet’s
autoloader when running the specs.</p>
<p>Next, we need to configure rspec-puppet, so create <code class="highlighter-rouge">spec/spec_helper.rb</code> with
the following contents</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="code"><pre><span class="nb">require</span> <span class="s1">'rspec-puppet'</span>
<span class="n">fixture_path</span> <span class="o">=</span> <span class="no">File</span><span class="p">.</span><span class="nf">expand_path</span><span class="p">(</span><span class="no">File</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="kp">__FILE__</span><span class="p">,</span> <span class="s1">'..'</span><span class="p">,</span> <span class="s1">'..'</span><span class="p">,</span> <span class="s1">'fixtures'</span><span class="p">))</span>
<span class="no">RSpec</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">c</span><span class="o">|</span>
<span class="n">c</span><span class="p">.</span><span class="nf">module_path</span> <span class="o">=</span> <span class="no">File</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="n">fixture_path</span><span class="p">,</span> <span class="s1">'modules'</span><span class="p">)</span>
<span class="n">c</span><span class="p">.</span><span class="nf">manifest_dir</span> <span class="o">=</span> <span class="no">File</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="n">fixture_path</span><span class="p">,</span> <span class="s1">'manifests'</span><span class="p">)</span>
<span class="k">end</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Now, all we need is a Rake task to fire up the tests. Create a <code class="highlighter-rouge">Rakefile</code> in
the root directory of your module with the following contents</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="code"><pre><span class="nb">require</span> <span class="s1">'rake'</span>
<span class="nb">require</span> <span class="s1">'rspec/core/rake_task'</span>
<span class="no">RSpec</span><span class="o">::</span><span class="no">Core</span><span class="o">::</span><span class="no">RakeTask</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">:spec</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">t</span><span class="o">|</span>
<span class="n">t</span><span class="p">.</span><span class="nf">pattern</span> <span class="o">=</span> <span class="s1">'spec/*/*_spec.rb'</span>
<span class="k">end</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>You can now run <code class="highlighter-rouge">rake spec</code> to run your <code class="highlighter-rouge">rspec-puppet</code> tests</p>
<h2 id="puppet-lint">puppet-lint</h2>
<p>Next up, we’ll also get some automatic lint testing of your manifests going to
ensure you’re writing manifests that comply with the Puppet Labs style guide.
This is simply a matter adding the following line near the top of your
<code class="highlighter-rouge">Rakefile</code></p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
</pre></td><td class="code"><pre><span class="nb">require</span> <span class="s1">'puppet-lint/tasks/puppet-lint'</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>You can now run <code class="highlighter-rouge">rake lint</code> to run puppet-lint over your manifests.</p>
<h2 id="travis-ci">Travis CI</h2>
<p><a href="https://travis-ci.org">Travis CI</a> is a wonderful free continuous integration
service that integrates with <a href="https://github.com">GitHub</a>, running whatever
tests you want against your code every time you push.</p>
<p>To get Travis CI automatically testing your module you need to add a couple of
files to the root directory of your module.</p>
<p>First, create a <code class="highlighter-rouge">Gemfile</code> which tells bundler which ruby gems your tests need in
order to run. If your module needs any additional gems, just add them to the
bottom of this file.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
</pre></td><td class="code"><pre><span class="n">source</span> <span class="ss">:rubygems</span>
<span class="k">if</span> <span class="no">ENV</span><span class="p">.</span><span class="nf">key?</span><span class="p">(</span><span class="s1">'PUPPET_VERSION'</span><span class="p">)</span>
<span class="n">puppetversion</span> <span class="o">=</span> <span class="s2">"= </span><span class="si">#{</span><span class="no">ENV</span><span class="p">[</span><span class="s1">'PUPPET_VERSION'</span><span class="p">]</span><span class="si">}</span><span class="s2">"</span>
<span class="k">else</span>
<span class="n">puppetversion</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'>= 2.7'</span><span class="p">]</span>
<span class="k">end</span>
<span class="n">gem</span> <span class="s1">'rake'</span>
<span class="n">gem</span> <span class="s1">'puppet-lint'</span>
<span class="n">gem</span> <span class="s1">'rspec-puppet'</span>
<span class="n">gem</span> <span class="s1">'puppet'</span><span class="p">,</span> <span class="n">puppetversion</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>We also need to create <code class="highlighter-rouge">.travis.yml</code> which holds our Travis CI test config.</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="code"><pre><span class="na">rvm</span><span class="pi">:</span> <span class="s">1.8.7</span>
<span class="na">notifications</span><span class="pi">:</span>
<span class="na">email</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s"><your email address></span>
<span class="na">env</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">PUPPET_VERSION=2.6.14</span>
<span class="pi">-</span> <span class="s">PUPPET_VERSION=2.7.11</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>This basically says, we want to use Ruby 1.8.7, run two sets of tests, one
against Puppet 2.6.14 and the other against Puppet 2.7.11 and email the
notifications through to your email address.</p>
<p>There’s one last thing we need to do and that is create a default <code class="highlighter-rouge">rake</code> task
that runs both <code class="highlighter-rouge">rake spec</code> and <code class="highlighter-rouge">rake lint</code>. To do that, add the following to
the end of your <code class="highlighter-rouge">Rakefile</code>.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
</pre></td><td class="code"><pre><span class="n">task</span> <span class="ss">:default</span> <span class="o">=></span> <span class="p">[</span><span class="ss">:spec</span><span class="p">,</span> <span class="ss">:lint</span><span class="p">]</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>If you haven’t already done so, commit and push all this up to GitHub.</p>
<p>Point your browser <a href="https://travis-ci.org">Travis CI</a> and login with your GitHub
account. In your profile page, turn on tests for your module’s repository.</p>
<p><img src="https://img.skitch.com/20120302-e2y2xk2cxb6mwnuhhynrjfp7m8.jpg" alt="Turn on tests" /></p>
<p>And wait for them to run!</p>
<p><img src="https://img.skitch.com/20120302-txxietenui82dsxyjubajnqt2e.jpg" alt="Win" /></p>
<h2 id="tldr">TL;DR</h2>
<p>Go check out my <a href="https://github.com/rodjek/puppet-logrotate">logrotate module</a> for
a working example.</p>
Stop Writing Puppet Modules That Suck2011-12-27T00:00:00+00:00https://bombasticmonkey.com/2011/12/27/stop-writing-puppet-modules-that-suck/<p>Whenever I need to setup a new service on one of my hosts, the first thing I do
is head to <a href="https://forge.puppet.com">the forge</a> and
<a href="https://github.com">GitHub</a> to try and find a decent Puppet module that
already exists for it.</p>
<p>I almost always leave in disappointment.</p>
<h2 id="puppet-modules-are-libraries">Puppet modules are libraries</h2>
<p>Much like <code class="highlighter-rouge">string.h</code> provides everything you need to manipulate strings in C,
your Puppet modules should provide everything needed to manage a service out of
the box. By that I mean, I want to pull down your module to enable the
functionality I need in Puppet <strong>without modifying your module at all</strong>.</p>
<h3 id="package-file-service">Package, File, Service</h3>
<p>Regrettably, most of the modules out there don’t deviate from the basic
<code class="highlighter-rouge">package</code>, <code class="highlighter-rouge">file</code>, <code class="highlighter-rouge">service</code> model.</p>
<figure class="highlight"><pre><code class="language-puppet" data-lang="puppet"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
</pre></td><td class="code"><pre><span class="k">class</span> <span class="nc">ntp</span> <span class="p">{</span>
<span class="n">package</span> <span class="p">{</span> <span class="s1">'ntp'</span><span class="p">:</span>
<span class="py">ensure</span> <span class="p">=></span> <span class="n">installed</span><span class="p">,</span>
<span class="p">}</span>
<span class="n">file</span> <span class="p">{</span> <span class="s1">'/etc/ntp.conf'</span><span class="p">:</span>
<span class="py">ensure</span> <span class="p">=></span> <span class="n">file</span><span class="p">,</span>
<span class="py">source</span> <span class="p">=></span> <span class="s1">'puppet:///modules/ntp/etc/ntp.conf'</span><span class="p">,</span>
<span class="py">owner</span> <span class="p">=></span> <span class="s1">'root'</span><span class="p">,</span>
<span class="py">group</span> <span class="p">=></span> <span class="s1">'root'</span><span class="p">,</span>
<span class="py">mode</span> <span class="p">=></span> <span class="s1">'0444'</span><span class="p">,</span>
<span class="kp">require</span> <span class="p">=></span> <span class="nc">Package</span><span class="p">[</span><span class="s1">'ntp'</span><span class="p">],</span>
<span class="kp">notify</span> <span class="p">=></span> <span class="nc">Service</span><span class="p">[</span><span class="s1">'ntp'</span><span class="p">],</span>
<span class="p">}</span>
<span class="n">service</span> <span class="p">{</span> <span class="s1">'ntp'</span><span class="p">:</span>
<span class="py">ensure</span> <span class="p">=></span> <span class="s1">'running'</span><span class="p">,</span>
<span class="py">enable</span> <span class="p">=></span> <span class="s1">'true'</span><span class="p">,</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>While this might be all you need in your homogeneous environment, it’s
unlikely that this will work in someone elses environment without
modifications.</p>
<h3 id="package-file-service-facter">Package, File, Service, Facter</h3>
<p>A common extension to this model is to add conditionals using Facter variables
to cover a set of different use cases (CentOS vs Debian, etc).</p>
<figure class="highlight"><pre><code class="language-puppet" data-lang="puppet"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
</pre></td><td class="code"><pre><span class="k">class</span> <span class="nc">ntp</span> <span class="p">{</span>
<span class="k">case</span> <span class="nv">$::operatingsystem</span> <span class="p">{</span>
<span class="nc">Debian</span><span class="p">:</span> <span class="p">{</span>
<span class="nv">$packagename</span> <span class="o">=</span> <span class="s1">'openntp'</span>
<span class="nv">$servicename</span> <span class="o">=</span> <span class="s1">'openntp'</span>
<span class="p">}</span>
<span class="k">default</span><span class="p">:</span> <span class="p">{</span>
<span class="nv">$packagename</span> <span class="o">=</span> <span class="s1">'ntp'</span>
<span class="nv">$servicename</span> <span class="o">=</span> <span class="s1">'ntp'</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">package</span> <span class="p">{</span> <span class="nv">$packagename</span><span class="p">:</span>
<span class="py">ensure</span> <span class="p">=></span> <span class="n">installed</span><span class="p">,</span>
<span class="p">}</span>
<span class="n">file</span> <span class="p">{</span> <span class="s1">'/etc/ntp.conf'</span><span class="p">:</span>
<span class="py">ensure</span> <span class="p">=></span> <span class="n">file</span><span class="p">,</span>
<span class="py">source</span> <span class="p">=></span> <span class="s1">'puppet:///modules/ntp/etc/ntp.conf'</span><span class="p">,</span>
<span class="py">owner</span> <span class="p">=></span> <span class="s1">'root'</span><span class="p">,</span>
<span class="py">group</span> <span class="p">=></span> <span class="s1">'root'</span><span class="p">,</span>
<span class="py">mode</span> <span class="p">=></span> <span class="s1">'0444'</span><span class="p">,</span>
<span class="kp">require</span> <span class="p">=></span> <span class="nc">Package</span><span class="p">[</span><span class="nv">$packagename</span><span class="p">],</span>
<span class="kp">notify</span> <span class="p">=></span> <span class="nc">Service</span><span class="p">[</span><span class="nv">$servicename</span><span class="p">],</span>
<span class="p">}</span>
<span class="n">service</span> <span class="p">{</span> <span class="nv">$servicename</span><span class="p">:</span>
<span class="py">ensure</span> <span class="p">=></span> <span class="s1">'running'</span><span class="p">,</span>
<span class="py">enable</span> <span class="p">=></span> <span class="kc">true</span><span class="p">,</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>While this is slightly more useful, it doesn’t cover the possibility of wanting
to override a value on a per-host basis.</p>
<h3 id="package-file-service-facter-and-global-variables--seriously">Package, File, Service, Facter and… Global Variables? Seriously?</h3>
<p>If you ask most people for the “best practice” method of passing parameters to
their modules, they’ll probably recommend using global variables - where you
set a variable in your node definition or ENC and it magically gets picked up
inside the various classes that you’ve included.</p>
<figure class="highlight"><pre><code class="language-puppet" data-lang="puppet"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
</pre></td><td class="code"><pre><span class="k">node</span> <span class="s1">'foo.example.com'</span> <span class="p">{</span>
<span class="nv">$ntp_running</span> <span class="o">=</span> <span class="kc">true</span>
<span class="nv">$monitor</span> <span class="o">=</span> <span class="kc">true</span>
<span class="nv">$backup</span> <span class="o">=</span> <span class="kc">true</span>
<span class="nv">$fml</span> <span class="o">=</span> <span class="kc">true</span>
<span class="k">include</span> <span class="nc">ntp</span>
<span class="p">}</span>
<span class="k">class</span> <span class="nc">ntp</span> <span class="p">{</span>
<span class="c"># trimmed for brevity
</span>
<span class="n">service</span> <span class="p">{</span> <span class="nv">$servicename</span><span class="p">:</span>
<span class="py">ensure</span> <span class="p">=></span> <span class="nv">$ntp_running</span> <span class="err">?</span> <span class="p">{</span>
<span class="py">true</span> <span class="p">=></span> <span class="s1">'running'</span><span class="p">,</span>
<span class="py">false</span> <span class="p">=></span> <span class="s1">'stopped'</span><span class="p">,</span>
<span class="p">},</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Do we really need to go into why this is a bad idea? Puppet’s variable scoping
is confusing enough on it’s own.</p>
<h2 id="so-what-should-you-be-doing">So what should you be doing</h2>
<p>To put it simply, write your Puppet modules like you would write a library for
your favorite programming language. Don’t know a programming language? You’re
working towards the almighty “Infrastructure as Code” ideal. Stop making
excuses like “I don’t know how to program so I didn’t know any better” and go
and learn a language.</p>
<ul>
<li>Don’t make your users edit your code to do the work they want.</li>
<li>Don’t rely on global variables to pass information to your modules.</li>
<li>Make it easy for other people to add <strong>features</strong> to your module.</li>
</ul>
<p>While it looks like the 3rd point contradicts the 1st point, it doesn’t.
There’s a big difference between someone sending you a patch to your module to
support a distro that it doesn’t currently support and someone having to go and
edit a template inside your module in order to change a config value that you
didn’t think anyone would need to change.</p>
<h2 id="an-example-of-a-good-module">An example of a good module</h2>
<p>Take a look at the Puppet Labs <a href="https://github.com/puppetlabs/puppetlabs-ntp/blob/master/manifests/init.pp">NTP
module</a>.
It’s not perfect, but it’s pretty close.</p>
<p>First off, it’s using a parameterised class, so there’s no global variables
polluting the namespace while still allowing us to change the parameters used
to configure NTP without editing the module itself. If you’re using an older
version of Puppet that doesn’t support parameterised classes, a defined type
will work just as well.</p>
<p>Secondly, it has support for a good number of different distributions <em>but</em> it
has a default case that causes Puppet to abort and let the user know that this
module isn’t supported on their machine. This is much better than just blindly
making changes to a system using values that may or may not work for it.</p>
<p>For the folks using ENCs that don’t support parameterised classes or defined
types, you’re still OK because you can just put this into your role definition
like so</p>
<figure class="highlight"><pre><code class="language-puppet" data-lang="puppet"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="code"><pre><span class="k">class</span> <span class="nc">mycompany::role::frontend</span> <span class="p">{</span>
<span class="k">class</span> <span class="p">{</span> <span class="s1">'ntp'</span><span class="p">:</span>
<span class="py">servers</span> <span class="p">=></span> <span class="p">[</span><span class="s1">'ntp.mycompany.com'</span><span class="p">],</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<h3 id="how-this-module-could-be-made-even-better">How this module could be made even better</h3>
<p>The addition of a generic <code class="highlighter-rouge">ntp::config</code> type that used either Augeas or
parsedfile on the back end to set arbitrary configuration values, i.e.</p>
<figure class="highlight"><pre><code class="language-puppet" data-lang="puppet"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="code"><pre><span class="k">class</span> <span class="p">{</span> <span class="s1">'ntp'</span><span class="p">:</span>
<span class="py">servers</span> <span class="p">=></span> <span class="p">[</span><span class="s1">'ntp.example.com'</span><span class="p">],</span>
<span class="p">}</span>
<span class="n">ntp::config</span> <span class="p">{</span>
<span class="s1">'statsdir'</span><span class="p">:</span>
<span class="py">value</span> <span class="p">=></span> <span class="s1">'/var/log/ntp'</span><span class="p">;</span>
<span class="s1">'statistics'</span><span class="p">:</span>
<span class="py">value</span> <span class="p">=></span> <span class="s1">'rawstats'</span><span class="p">;</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Also, replacing the <code class="highlighter-rouge">autoupdate</code> parameter with a <code class="highlighter-rouge">version</code> parameter so that
it was possible to specify a particular package version, ‘latest’, ‘installed’
etc. would be a great addition.</p>
<h2 id="thats-all-folks">That’s all folks</h2>
<p>How would you like to see modules written? What currently annoys you about the
modules out there? Do you disagree with everything I’ve written and want the
last 10 minutes of your life back? Let me know.</p>
Auto-notify Resources From Your Puppet Types2011-11-13T00:00:00+00:00https://bombasticmonkey.com/2011/11/13/autonotify-resources-from-your-puppet-types/<p>So, I ran into a bit of a problem yesterday. I was working on a Puppet
module for the <a href="https://github.com/sonian/sensu/">Sensu</a> monitoring framework,
part of which involved writing a custom Puppet type to manage the configuration
of service checks. Normally this is a trivial matter, however in this case,
the checks must be configured on both the server <strong>and</strong> the client and must
therefore be able to notify either the server process or the client process or
both to restart after a configuration change.</p>
<p>Obviously, I couldn’t just do this:</p>
<figure class="highlight"><pre><code class="language-puppet" data-lang="puppet"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="code"><pre><span class="n">sensu_check</span> <span class="p">{</span> <span class="s1">'mycheck'</span><span class="p">:</span>
<span class="kp">notify</span> <span class="p">=></span> <span class="p">[</span>
<span class="nc">Service</span><span class="p">[</span><span class="s1">'sensu-client'</span><span class="p">],</span>
<span class="nc">Service</span><span class="p">[</span><span class="s1">'sensu-server'</span><span class="p">],</span>
<span class="p">],</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>As if the host was only running the client process, this would result in an
error during the Puppet runs.</p>
<p>One option would be to use the following mess:</p>
<figure class="highlight"><pre><code class="language-puppet" data-lang="puppet"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
</pre></td><td class="code"><pre><span class="nf">if</span><span class="p">(</span><span class="nf">defined</span><span class="p">(</span><span class="nc">Service</span><span class="p">[</span><span class="s1">'sensu-client'</span><span class="p">])</span> <span class="err">&&</span> <span class="nf">defined</span><span class="p">(</span><span class="nc">Service</span><span class="p">[</span><span class="s1">'sensu-server'</span><span class="p">]))</span> <span class="p">{</span>
<span class="nc">Sensu_check</span> <span class="p">{</span>
<span class="kp">notify</span> <span class="p">=></span> <span class="p">[</span>
<span class="nc">Service</span><span class="p">[</span><span class="s1">'sensu-client'</span><span class="p">],</span>
<span class="nc">Service</span><span class="p">[</span><span class="s1">'sensu-server'</span><span class="p">],</span>
<span class="p">],</span>
<span class="p">}</span>
<span class="p">}</span> <span class="k">elsif</span> <span class="nf">defined</span><span class="p">(</span><span class="nc">Service</span><span class="p">[</span><span class="s1">'sensu-client'</span><span class="p">])</span> <span class="p">{</span>
<span class="nc">Sensu_check</span> <span class="p">{</span>
<span class="kp">notify</span> <span class="p">=></span> <span class="nc">Service</span><span class="p">[</span><span class="s1">'sensu-client'</span><span class="p">],</span>
<span class="p">}</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="nc">Sensu_check</span> <span class="p">{</span>
<span class="kp">notify</span> <span class="p">=></span> <span class="nc">Service</span><span class="p">[</span><span class="s1">'sensu-server'</span><span class="p">],</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Not the most elegant solution in the world. Ideally what I needed was
something like the existing <code class="highlighter-rouge">autorequire</code> functionality that creates <code class="highlighter-rouge">require</code>
relationships between your custom type and the named resources automatically
but only if the resources exist in the node catalogue.</p>
<p>Unfortunately, such functionality doesn’t exist (yet), so I had to go some
digging around in Puppet’s codebase to implement it. The first thing to do was
to establish a way to create arbitrary <code class="highlighter-rouge">notify</code> relationships. This turned out
to be easily done:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="code"><pre><span class="no">Puppet</span><span class="o">::</span><span class="no">Type</span><span class="p">.</span><span class="nf">newtype</span><span class="p">(</span><span class="ss">:mytype</span><span class="p">)</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="k">super</span>
<span class="nb">self</span><span class="p">[</span><span class="ss">:notify</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span>
<span class="s2">"Service[myservice]"</span><span class="p">,</span>
<span class="s2">"Service[myotherservice]"</span><span class="p">,</span>
<span class="p">]</span>
<span class="k">end</span>
<span class="k">end</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Now, all I needed to do was make it so that it only notified the resources that
actually existed in the node’s manifest</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="code"><pre><span class="no">Puppet</span><span class="o">::</span><span class="no">Type</span><span class="p">.</span><span class="nf">newtype</span><span class="p">(</span><span class="ss">:mytype</span><span class="p">)</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="k">super</span>
<span class="nb">self</span><span class="p">[</span><span class="ss">:notify</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span>
<span class="s2">"Service[myservice]"</span><span class="p">,</span>
<span class="s2">"Service[myotherservice]"</span><span class="p">,</span>
<span class="p">].</span><span class="nf">select</span> <span class="p">{</span> <span class="o">|</span><span class="n">ref</span><span class="o">|</span> <span class="n">catalog</span><span class="p">.</span><span class="nf">resource</span><span class="p">(</span><span class="n">ref</span><span class="p">)</span> <span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>The moral of this extremely drawn out story? Anything is possible in Puppet if
you’re willing to get your hands dirty. And this should really be a built in
feature *hint*.</p>
Test Your Puppet Modules - Functions2011-11-04T00:00:00+00:00https://bombasticmonkey.com/2011/11/04/test-your-puppet-modules-functions/
<figure class="highlight"><pre><code class="language-irc" data-lang="irc"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="code"><pre>11:00 <hubot> [puppet/master] merge branch 'mysql-module-refactor' - dave
11:00 <hubot> dave is deploying puppet/master to production
11:23 <nagios> PROBLEM - MySQL on dbmaster1.initech.com is CRITICAL
11:24 <dave> oh fuck
</pre></td></tr></tbody></table></code></pre></figure>
<p>Look familiar?</p>
<p>If you’ve worked with configuration management systems for a while, a situation
like this has probably cropped up and ruined your day. If you’re lucky
enough to have a homogeneous environment, then you might have a staging
environment that you can test changes out on, but what if you don’t? A slight
mistake in that harmless change you’re working on could stop a service on
hundreds of machines or worse (purge the mysql-server package and all it’s data
*cough*).</p>
<h2 id="why-you-should-be-writing-unit-tests-for-your-puppet-modules">Why you should be writing unit tests for your Puppet modules</h2>
<ul>
<li>Prevent situations like the one above.</li>
<li>Catch any problems moving between Puppet releases before it hits production.</li>
<li>Now we can do this too</li>
</ul>
<p><img src="https://imgs.xkcd.com/comics/compiling.png" alt="Obligatory XKCD" /></p>
<h2 id="getting-started">Getting started</h2>
<p>In this article, we’ll cover setting up your testing environment and cover how
to write unit tests for your Puppet functions. I’m going to make a few
assumptions now:</p>
<ul>
<li>You store your Puppet manifests in git.</li>
<li>You run a *nix machine as your workstation.</li>
<li>You don’t mind getting your hands dirty with a bit of simple Ruby.</li>
<li>You have Ruby installed (1.8.7).</li>
</ul>
<p>First of all, we’re going to install <a href="https://bundler.io">Bundler</a> to manage
the dependencies our Puppet testing rig will have.</p>
<figure class="highlight"><pre><code class="language-console" data-lang="console"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="code"><pre><span class="gp">$</span><span class="w"> </span>gem <span class="nb">install </span>bundler <span class="nt">--no-ri</span> <span class="nt">--no-rdoc</span>
<span class="go">Fetching: bundler-1.0.21.gem (100%)
Successfully installed bundler-1.0.21
1 gem installed
</span><span class="gp">$</span><span class="w"> </span>bundle <span class="nt">--version</span>
<span class="go">Bundler version 1.0.21</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Next we’re going to create a <code class="highlighter-rouge">Gemfile</code> in the top level of our Puppet repo with
the list of gems we’re going to need. Adjust the <code class="highlighter-rouge">puppet</code> and <code class="highlighter-rouge">facter</code>
versions to match your environment.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="code"><pre><span class="n">source</span> <span class="ss">:rubygems</span>
<span class="n">gem</span> <span class="s1">'puppet'</span><span class="p">,</span> <span class="s1">'2.6.12'</span>
<span class="n">gem</span> <span class="s1">'facter'</span><span class="p">,</span> <span class="s1">'1.6.0'</span>
<span class="n">gem</span> <span class="s1">'rspec-puppet'</span><span class="p">,</span> <span class="s1">'0.1.0'</span>
<span class="n">gem</span> <span class="s1">'rake'</span><span class="p">,</span> <span class="s1">'0.8.7'</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Next we’re going to add a couple of things to your <code class="highlighter-rouge">.gitignore</code>.</p>
<figure class="highlight"><pre><code class="language-text" data-lang="text"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
</pre></td><td class="code"><pre>vendor/gems/
.bundle/
</pre></td></tr></tbody></table></code></pre></figure>
<p>Now we just need to tell bundler to install everything and commit our
changes to the repository.</p>
<figure class="highlight"><pre><code class="language-console" data-lang="console"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
</pre></td><td class="code"><pre><span class="gp">$</span><span class="w"> </span>bundle <span class="nb">install</span> <span class="nt">--path</span> vendor/gems
<span class="go">Fetching source index for https://rubygems.org/
Installing rake (0.8.7)
Installing diff-lcs (1.1.3)
Installing facter (1.6.0)
Installing puppet (2.6.12)
Installing rspec-core (2.7.1)
Installing rspec-expectations (2.7.0)
Installing rspec-mocks (2.7.0)
Installing rspec (2.7.0)
Installing rspec-puppet (0.1.0)
Using bundler (1.0.21)
Your bundle is complete! It was installed into ./vendor/gems
</span><span class="gp">$</span><span class="w"> </span>git add Gemfile
<span class="gp">$</span><span class="w"> </span>git add Gemfile.lock
<span class="gp">$</span><span class="w"> </span>git commit <span class="nt">-a</span> <span class="nt">-m</span> <span class="s2">"Bundler setup for testing"</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>OK, time to configure <a href="https://www.relishapp.com/rspec">RSpec</a>. Create
a <code class="highlighter-rouge">spec</code> directory in the root of your Puppet repository and create a file in
the <code class="highlighter-rouge">spec</code> folder called <code class="highlighter-rouge">spec_helper.rb</code>. Adjust <code class="highlighter-rouge">c.module_path</code> and
<code class="highlighter-rouge">c.manifest_dir</code> to point to your modules and manifests directories in your
repository.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="code"><pre><span class="nb">require</span> <span class="s1">'rspec-puppet'</span>
<span class="no">RSpec</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">c</span><span class="o">|</span>
<span class="n">c</span><span class="p">.</span><span class="nf">module_path</span> <span class="o">=</span> <span class="s2">"modules"</span>
<span class="n">c</span><span class="p">.</span><span class="nf">manifest_dir</span> <span class="o">=</span> <span class="s1">'manifests'</span>
<span class="k">end</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>The last thing we need to do is create a <a href="https://ruby.github.io/rake">Rake</a> task
to run our tests. Create a <code class="highlighter-rouge">Rakefile</code> in the root of your Puppet repository
with the following.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="code"><pre><span class="nb">require</span> <span class="s1">'rake'</span>
<span class="nb">require</span> <span class="s1">'rspec/core/rake_task'</span>
<span class="no">RSpec</span><span class="o">::</span><span class="no">Core</span><span class="o">::</span><span class="no">RakeTask</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">:test</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">t</span><span class="o">|</span>
<span class="n">t</span><span class="p">.</span><span class="nf">pattern</span> <span class="o">=</span> <span class="s1">'modules/*/spec/*/*_spec.rb'</span>
<span class="k">end</span>
<span class="n">task</span> <span class="ss">:default</span> <span class="o">=></span> <span class="ss">:test</span>
</pre></td></tr></tbody></table></code></pre></figure>
<h2 id="testing-your-first-function">Testing your first function</h2>
<p>For the sake of this example, let’s create a simple module with a single
function (that I’m going to borrow from Puppet Lab’s stdlib module).</p>
<figure class="highlight"><pre><code class="language-console" data-lang="console"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
</pre></td><td class="code"><pre><span class="gp">$</span><span class="w"> </span><span class="nb">mkdir</span> <span class="nt">-p</span> modules/misc/lib/puppet/parser/functions
<span class="gp">$</span><span class="w"> </span><span class="nb">mkdir</span> <span class="nt">-p</span> modules/spec/functions
</pre></td></tr></tbody></table></code></pre></figure>
<p>Download <a href="https://raw.github.com/puppetlabs/puppetlabs-stdlib/master/lib/puppet/parser/functions/bool2num.rb">this
function</a>
and drop it in <code class="highlighter-rouge">modules/misc/lib/puppet/parser/functions</code>.</p>
<p>Basically this function should</p>
<ul>
<li>return 0 if you pass it ‘f’, ‘false’, ‘n’, ‘no’, 0, ‘’, ‘undef’ or
‘undefined’.</li>
<li>return 1 if you pass it ‘t’, ‘true’, ‘y’, ‘yes’ or 1.</li>
<li>raise Puppet::ParseError if you pass it anything else.</li>
</ul>
<p>Before we continue, you should now go and read the <a href="https://github.com/rodjek/rspec-puppet/blob/master/README.md">rspec-puppet
README</a>. The
tests for this function should live in <code class="highlighter-rouge">modules/misc/spec/functions/bool2num_spec.rb</code>.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
</pre></td><td class="code"><pre><span class="nb">require</span> <span class="s1">'spec_helper'</span>
<span class="n">describe</span> <span class="s1">'bool2num'</span> <span class="k">do</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'true'</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'t'</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'y'</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'yes'</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'1'</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'false'</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'f'</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'n'</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'no'</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'0'</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'undef'</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'undefined'</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">''</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="p">{</span> <span class="n">should</span> <span class="n">run</span><span class="p">.</span><span class="nf">with_params</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">).</span><span class="nf">and_raise_error</span><span class="p">(</span><span class="no">Puppet</span><span class="o">::</span><span class="no">ParseError</span><span class="p">)</span> <span class="p">}</span>
<span class="k">end</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Most of this should be pretty self explanatory, however there is a couple of
important things:</p>
<ul>
<li>The <code class="highlighter-rouge">spec_helper.rb</code> required on line 1 is the <code class="highlighter-rouge">spec_helper.rb</code> in the
<code class="highlighter-rouge">spec</code> directory at the root of your repository, not from the per-module
<code class="highlighter-rouge">spec</code> directory.</li>
<li>The description on line 3 <strong>must</strong> be a string and it <strong>must</strong> be the name
of the function that you are testing so that RSpec can set the subject
correctly.</li>
</ul>
<p>Now for the all important running of the tests.</p>
<figure class="highlight"><pre><code class="language-console" data-lang="console"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="code"><pre><span class="gp">$</span><span class="w"> </span>bundle <span class="nb">exec </span>rake
<span class="go">/usr/bin/ruby -S rspec modules/misc/spec/functions/bool2num_spec.rb
</span><span class="c">..............
</span><span class="go">
Finished in 2.61 seconds
14 examples, 0 failures</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Huzzah! Now go and write tests for the rest of your functions. In the next
article in this series, we’ll cover how to test your Puppet manifests (<code class="highlighter-rouge">.pp</code>
files).</p>
Distributing Augeas lenses with Puppet's pluginsync2011-01-02T00:00:00+00:00https://bombasticmonkey.com/2011/01/02/distributing-augeas-lenses-with-pluginsync/<p>Sick of having to write <code class="highlighter-rouge">file</code> resources to distribute the Augeas lenses that
your module depends on? Why not use Puppet’s pluginsync functionality to
distribute them with the rest of your module?</p>
<p>Under your module’s <code class="highlighter-rouge">lib</code> directory, create the following directory structure</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>modules/<module>/lib/
augeas/
lenses/
</code></pre></div></div>
<p>Your module should now look like this:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>modules/<module>/manifests/
templates/
lib/
puppet/
facter/
augeas/
lenses/
</code></pre></div></div>
<p>Drop your Augeas lenses into this <code class="highlighter-rouge">lenses</code> directory and Puppet will
distribute them to all your clients automatically. Now we just need to tell
Augeas where to find these lenses.</p>
<p>The easiest way to go about this is to set a default <code class="highlighter-rouge">load_path</code> value for
Augeas type. To do that, add the following to your <code class="highlighter-rouge">site.pp</code>.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><table class="rouge-table"><tbody><tr><td class="gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="code"><pre><span class="no">Augeas</span> <span class="p">{</span>
<span class="n">load_path</span> <span class="o">=></span> <span class="s2">"/usr/share/augeas/lenses:${settings::vardir}/augeas/lenses"</span><span class="p">,</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></figure>
<p>Note: Puppet will display an error for each lens during runs as it’ll try to
load them as Ruby files. It’s noisy, but it’s not a fatal error. I’m hoping
to find an easy way to prevent this.</p>