• Persistent sequential Puppet

    by  • 19th February 2013 • post

    How damn annoying it is to try to get puppet agent to run in a predictable manner, not overloading master server.

    Scenario :

    • puppet agent gets to run every X minutes, we use cron to fire it up ( puppet agent –onetime )
    • cron is managed by puppet, naturally

    Now if you do lets say put */10 for the minute … it runs on the dot every 10min … not pretty … master suddenly gets hit by XXX amount of agents demanding attention …

    Smart way was to use $fqdn_rand(10) to get the offset … however we have found that random doesn’t mean good , out of 20 agents 9 were run on 0 minute … bad …

    Now comes a little solution in a custom function. Yeah, loads of puppet purists start vomiting now … but hey, it works like a charm!

    It’s a persistent sequencing function, using filesystem to store the values ( you can use some nosql stores as well – just modify it ) in the for of /tmp/seq-{name}/{max_number}/{fqdn}

    I’ve called it sequencer(max_number, name), the 2 arguments are :

    • max_number – so if you want to run puppet agent every 10m , you just say 10 here …
    • name – so that you can have names sequences for different things

    The script is available on bitbucket and you have to put it in lib/puppet/parser/functions/ as per docs on custom functions.

    and then in puppet manifest you would use it like that:

    $skey = sequencer(10,'agent')
    cron { "Puppet Agent":
    command => '/usr/bin/puppet agent --onetime',
    minute=> [$skey , $skey+10, $skey+20, $skey+30, $skey+40, $skey+50],
    user => 'root'
    }