Today I stumbled upon an unexpected behaviour change in CFEngine templates when upgrading from version 3.4 to 3.6. The change is not documented anywhere in the Changelog, so I am really not sure if it’s a bug or a feature. In any case, it is something to be aware of.
Take this template:
Normally, each line expands to an insert lines promise, which means that duplicated lines may not be printed more than once. That behavior has changed with CFEngine versions. [%CFEngine cfengine_3_4:: %] In this version of CFEngine, $(sys.cf_version), duplicates are printed once In this version of CFEngine, $(sys.cf_version), duplicates are printed once In this version of CFEngine, $(sys.cf_version), duplicates are printed once that holds for blank lines, too [%CFEngine cfengine_3_6:: %] In this version of CFEngine, $(sys.cf_version), duplicates are preserved In this version of CFEngine, $(sys.cf_version), duplicates are preserved In this version of CFEngine, $(sys.cf_version), duplicates are preserved [%CFEngine any:: %] End of the story!
and this policy:
bundle agent test_example_cf3_template
{
vars:
cfengine_3_4::
"templatedir" string => execresult("/bin/pwd","noshell") ;
"testfile" string => "/tmp/cf_34.txt" ;
cfengine_3_6::
"templatedir" string => "$(this.promise_dirname)" ;
"testfile" string => "/tmp/cf_36.txt" ;
files:
"$(testfile)"
create => "yes",
edit_template => "$(templatedir)/example_cf3_template.tmpl";
reports:
cfengine_3::
"Check output in $(testfile)" ;
}
Now run the policy in both CFEngine 3.4 and 3.6. You’d expect to get two identical files: /tmp/cf_34.txt and /tmp/cf_36.txt. But they’re not.
When I run cf-agent -Kf ./test_example_cf3_template.cf -b test_example_cf3_template in CFEngine 3.4 this is what /tmp/cf_34.txt looks like:
Normally, each line expands to an insert lines promise, which means that duplicated lines may not be printed more than once. That behavior has changed with CFEngine versions. In this version of CFEngine, 3.4.5, duplicates are printed once that holds for blank lines, too End of the story!
and when I run it in 3.6 this is what /tmp/cf_36.txt looks like:
Normally, each line expands to an insert lines promise, which means that duplicated lines may not be printed more than once. That behavior has changed with CFEngine versions. In this version of CFEngine, 3.6.4, duplicates are preserved In this version of CFEngine, 3.6.4, duplicates are preserved In this version of CFEngine, 3.6.4, duplicates are preserved End of the story!
What happened? I don’t know honestly, but by the looks of it the templates in 3.6 have escaped the insert_lines approach and are going for something different. The guys at CFEngine can tell us more, but meanwhile beware: if you relied on the behaviour from 3.4 you have to review your templates carefully.
A sysadmin's logbook 
There are a number of problems with templates. To workaround repeating lines or even truncated output, I put the cfengine BEGIN/END block tags around all the text between the cfengine class tags. It is messy, but I’ve had no problems with that style. You also have to make sure no block is bigger than 4K bytes (just end and start a new block, before the 4K limit).
I’d argue that this was a long-standing bug that was fixed.
When I’m using a template, I expect it to be precisely that — a template. Something that would be reproduced, with variables transformed as needed.
edit_template under 3.4 was unusable (segfaulted in odd places, nondeterministic output).
3.5 would strip duplicate lines present in the template from the output, even if not surrounded by BEGIN/END block tags. This made 3.5 basically unusable for things like postfix main.cf, where the same directive would need to be present in different locations of the file. This also stripped whitespace padding, which seriously impacted readability.
With 3.6, what’s in the template is what goes into the output file. If there’s a duplicate line in the template, then it’s in the output file. 3.6, IMO, is performing as would be expected, and previous behavior was incorrect.
Just my two cents, though. I do hope this behavior doesn’t revert, as that would seriously impact my infrastructure …
Hi CK
I prefer this behaviour, too, and I share your point of view: that’s how templates should work. I haven’t seen any comments in the community that mark this behaviour as a regression, so I would guess that we are safe 😉
Thanks for commenting here, ciao!
— bronto