When I began using puppet, I quickly realised that configuration data was best kept separate from puppet manifests. Initially, I used extlookup and kept configuration data in CSV files. Then complex data structures came to puppet and I now use hirea/hiera-puppet with configuration data stored in hierarchical YAML files (other hiera backends are available). This article describes how to define in YAML the resources that should be applied to a node.

I have been using the hiera_include function for some time to define which classes are applied to each node. Usage is fairly simple. I have this fragment in nodes.pp:

node default {
    hiera_include('classes')
}

Then, in an hiera YAML file I have something like this:

classes:
    - role::monitoring
    - role::dns::master

At run time, the puppet classes role::monitoring and role::dns::master are applied to the node. Note, this could be in a node-specific config file, or something more generic that applied to several nodes.

This all works fine, but is limited to static classes. What if you have defined classes and you want to apply multiple instances of the same defined class on one node?

Let's suppose you have a puppet define that puts an apache website configuration on a node:


 
define apache::website::instance(
    $ListenIP,
    $ServerAlias = [],
    $config = ''
) {
# clever code goes here to instantiate the apache vhost
}

You would use this define something like this:

apache::website::instance{'www.example.com':
    ListenIP    => '1.2.3.4',
    ServerAlias => ['example.com', 'foo.example.com'],
    config      => 'DocumentRoot /var/www/sites/example
<Directory /var/www/sites/example/>
    AllowOverride None
    Options +Indexes
</Directory>'
}

With the addition of a new custom puppet function it becomes possible to define the previous example in a YAML file.

Put the following content in $PUPPET_MODULE_DIR/custom/lib/puppet/parser/functions/hiera_resources.rb:

module Puppet::Parser::Functions
  newfunction(:hiera_resources, :type => :statement) do |args|
    raise Puppet::Error, "hiera_resources requires 1 argument; got #{args.length}" if args.length != 1
    res_name = args[0][0]
    apps = function_hiera_hash([res_name, {}])
    apps.each { | res_type, res_params | function_create_resources([res_type, res_params]) }
  end
end

And add this to nodes.pp:

node default {
    hiera_resources('resources')
}

Now, to create the resource as in the previous example, add this to the hiera config:

resources:
    apache::website::instance:
        'www.example.com':
            ListenIP: '1.2.3.4'
            ServerAlias:
                - 'example.com'
                - 'foo.example.com'
            config: |-
                DocumentRoot /var/www/sites/example
                <Directory /var/www/sites/example/>
                    AllowOverride None
                    Options +Indexes
                </Directory>

You can now add resource to nodes dynamically by editing your hiera config!

Thanks to:

  • Volcane, for writing extlookup and hiera
  • Hunner, for helping with the custom function (I say "help"… he wrote it!)

7 thoughts on “Assigning resources to nodes with hiera in puppet

  1. I am not sure I am grasping the benefits of that really. Why not have a websites module and define website::example-com as a class then apply that to the server? It seems to be just another layer of abstraction. Could you maybe post something about what benefits moving all this outside puppet gains?
    Thanks
    john

  2. John,

    For only one resource it is indeed not worth it. However, the specific use-case I have developed this for is the deployment of multiple jboss servers on one node. So, I will write one define that deploys a new jboss server, and invoke it from the hiera config by putting something like this in a node YAML file:

    resources:
        app::core::instance:
             core01:
                 index: 01
             core02:
                 index: 02
             core03:
                 index: 03
             core07:
                 index: 07

    Does that make more sense?

  3. One thing you might run into – there is a bug in some versions of hiera_hash which causes it to fail if no resources are found for a node. It shouldn't do this as the call to hiera_hash in hiera_resources specifies a default of an empty hash, but the bug causes hiera_hash to throw a "Could not find data item resources in any Hiera data file and no default supplied".

    I fixed this by hacking hiera_hash. Just a single line fixes it:

     

            answer = hiera.lookup(key, default, hiera_scope, override, :hash)
            return answer if answer == default
            raise(Puppet::ParseError, "Could not find data item #{key} in any Hiera data file and no default supplied") if answer.empty?
  4. Nice posting, but I can’t get this to run.

    config on my dev master:
    – ubuntu latest upstream packages and official packages from puppetlabs included
    – hiera: 1.2.1
    – ruby: ruby 1.8.7 (2010-01-10 patchlevel 249) [x86_64-linux]
    – libapache2-mod-passenger: 2.2.7
    – puppet v3.2.1

    The agent-run on the target machine (client) returns error: Error 400 on SERVER: Puppet::Parser::AST::Resource failed with error ArgumentError: malformed format string – %S at […]
    When I remove the yaml file from masters’ hieradata directory, the agent run on the target node is working. So my conclusion is, that the problem is within the yaml file

    resources:
    bcs_deployment::classes::instance:
    'inst1':
    ensure: 'running'
    version: '7.6.12'
    edition: 'full'

    I also tried removing the quotes.

    At this moment the configuration does not contain a call to your custom function “hiera_resources”, yet.
    As I’m not familiar with ruby, I absolutely have no idea, where to search for the problem.

    Would you help me?

    Regards
    Bjoern

  5. I have made little modifications to be able to reuse a defined type: hiera_resources.rb

    The problem was that I want to define the same defined type (eg.: check_tcp) several times in different hiera files.
    But as you define the defined type in the first place, hiera only returns the first match, so I changed it, and define first a descriptive name and then the defined type.

    Example:

    “mon_commands” : {
    “notify-host-by-email” : {
    “monitorizacion::icinga::commands” : {
    “command_line” : “/usr/bin/printf no definido”
    }
    },
    “notify-service-by-email” : {
    “monitorizacion::icinga::commands” : {
    “command_line” : “/usr/bin/printf no definido”
    }
    }

Leave a reply

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> 

required