Yeah I know, I shouldn't say it but sometimes a man just needs let his feelings be known.
I'm new to the whole DevOps sphere and release engineering etc. I am a quasi experienced developer who has had a little bit of experience with unit testing and test driven development. I don't want to blow my own trumpet here guys but I did in fact buy a copy of "THE ART OF UNIT TESTING" and whats more one day I will read it*
So when working on my current project, which is my first foray into DevOps and Chef I was quite enthralled by the concept of testing all this good stuff.
My little head buzzing with yet more buzzwords a heady mix of infastructure as code coupled with test driven development. Things are going be great lads we're going to finally be able to TEST ALL OF THE THINGS.
So I cracked open a copy of Test-Driven Infastructure with Chef and I looked at minitest-handler and I looked at Chef Spec and I thought yeah there's some lovely things going on here. There's some great state checking/functional testing going on with these tools and I like it but something was nagging away at me. What about unit testing?
Now whenever I did my brief toe in the water searches on this I was getting more confused. Whenever someone asked for a good unit testing framework they were being pointed towards these tools almost exclusively. Odder still nobody was really correcting them or saying no that's not quite what I'm after. So that begs the questions right, have I gone mad? What is Unit Testing with Chef?
Well let's start with what it's not, or at least what I think it's not. I don't believe simply checking the state of the box after your runlist has executed cuts it. That's a functional test and you are treating your chef code as a blackbox right? That's great that after running 50 recipes I know that 7 services have been created but do I really know that my code underlying is executing correctly?
So it seems very quickly that we've gone from TEST ALL OF THE THINGS to TEST WHAT ALL OF THE THINGS HAVE DONE WHEN THEY ARE FINISHED AND HOPE THEY AREN'T USING THE SAME RESOURCES or put another TEST ALL OF THE THINGS BY ASSERTING THE POSITIVE PATH
To me I feel like if I have crafted a provider or a library, then I should be looking to unit test those in the same way that the code I'm deploying has been tested. Is that too much? If I have a library with 20 methods I should have tests for each method executing as much of the code path as I think is appropriate.
So what I did was start looking at what happens during a node converge when a the cookbooks/methods/providers get loaded for execution and I tried to do the same from within a minitest file. Below is my attempt at testing the registry_helper library within the windows cookbook as I had noticed some funk with w2008r2 servers
require 'chef'
require "minitest/autorun"
require 'win32/registry'
require 'ruby-wmi'
class RegistryTest < MiniTest::Unit::TestCase
def setup
Kernel.load(File.expand_path(File.dirname(__FILE__))<< "\\..\\..\\..\\..\\libraries\\registry_helper.rb")
end
def test_GetHiveNameLM
hivename = Registry.get_hive_name("HKLM\\SOFTWARE\\Wow6432Node\\Clients\\Mail")
assert_equal "HKEY_LOCAL_MACHINE", hivename
end
So in the setup we're loading the library and then we can access methods defined within it and check the responses coming back. Similar functionality could be exercised in a test.rb within the cookbook but I think we would lose the ability to have asserts and metrics displayed. Some more complicated functions such as managing node objects or resources when testing providers can be exercised as well. Example shown below
node = Chef::Node.new
run_context = Chef::RunContext.new(node, Chef::CookbookCollection.new)
resource = Chef::Resource::ResourceName.new("UnitTest",run_context)
resource.version("version")
provider = Chef::Provider::ResourceName.new(resource, run_context)
provider.uuid = "123"
result = provider.dosomestuff()
Another potential advantage over having this as a test.rb is that we could enforce unit test runs before allowing these cookbooks to be uploaded meaning that broken or ineffective cookbooks never end up in the deployment chain. This is similar to function used on most CI systems with the code which we are deploying.
It's not perfect and it's not pretty but it's allowing me to drive specific methods from my libraries and provider.
Thoughts? Comments?
*I jest I did read it and it was a very good read. Much recommended
