Content
Config Based Automation
What is config based automation?
In infrastructure management, config-based automation is the norm. It means that the desired state of an infrastructure is described in a config file. How this desired state is achieved is hidden from the user. This has the huge advantage of elegance and simplicity: a lot of complexity is hidden from the user, who simply thinks about what desired state she wants to achieve and doesn’t have to care about how to create it.
What are the advantages of config based automation?
Main advantages are:
- Checking the current state and deciding which steps need to be taken in order to get from the current to the desired state is taken care of by the automation tool. The user doesn’t have to think about this at all. This is a huge simplification, since a myriad of different current states can require a myriad different individual steps and processes to get to a desired state.
- Order of operations is also taken care of by the automation tool and the user doesn’t have to worry about it (mostly).
What are the disadvantages of config based automation?
Letting the automation tool take care of the underlying logic of how to create a desired state is handy, but also means that the user has no control over it. This can be impractical, for example:
- Order or operations might be significant and not possible to define in the config file. For example, deploying a software with a DB, backend and frontend component might require the DB component to be up and running before the backend and frontend can be deployed, since they might expect the DB backend to be up and crash if they are deployed first. Any logic of this kind which the automation tool doesn’t know about can also not be defined in the config file and might require some hacks, or simply not be possible to do with a config based automation tool.
- Other types of logic might be required or handy when defining a deployment process. For example, several versions of a library could be used and an order of preference should be defined – often not possible with a config-based tool. Definitions of fallbacks are also something that might not be possible with a config-based tool – or it can be possible, which then already presents a kind of “hybrid” approach where the config-based tool allows for some logic, but only in very narrow cases.
In addition, there are a number of common and useful features that are very simple to implement in a code-based automation tool but will require the use of an additional tool when using a config-based automation tool, for example:
- Scheduling, notifications, custom error handling
- Dynamic creation of configurations, for example fetching the IP address of a VM to which a software should be deployed from a VM management tool, or fetching the domain name for a service from a database.
Configuration drift
A major drawback of config-based automation tools is also the difficulty of managing a large number of config files which multiply quickly. The possible permutations of configuration options are huge, even with simple systems that need little configuration. This leads to many configuration files with small differences, often distributed across a large number of locations. For example, each developer might have their own set of several such config files for different environments they use, spread across several of their projects in their source control system. Any change in deployment configuration means that some changes may need to be applied to all, or only a specific subset of these files – which is horrible to manage when there are lots, it is difficult to identify the ones that need the fix applied, and they are spread around a lot of different systems. The pain and time invested in the maintenance of these files often leads to initiatives to manage them sensibly, often by automating creation and changes of config files. This, however, requires the introduction of yet another tool or several tools to enable this. Often, this defeats the purpose by introducing an additional layer of complexity – when it would have been simpler to use a code-based tool from the get-go.
Code Based Automation
What is code based automation?
Code based automation is exactly what it sounds like: automation done using code. It uses code to define a process to get from a current state to a desired state.
Programming is in essence code based automation. However what I mean here are code-based automation tools, which, like config-based automation tools, provide some functionality around the code to make automation using code easier than simply writing a script from scratch.
What are the advantages of code based automation?
The advantages of code based automation are:
- Control over every last detail
- Ability to both define the desired state as well as the specific steps to get there from any current state
Code based automation can of course also use configuration files, so it is possible to define the desired state in a very structured way, while still retaining full control over how this desired state is reached. Everything that can be achieved with config based automation can also be achieved with code based automation, and more.
In addition, a lot of housekeeping is simple to do with code based automation which is simply not possible with config based automation, such as scheduling, notifications, custom error handling etc. However I have to point out that it is bad practice to include something like an error notification within a piece of code that defines an automated process, since it is a “meta-function” that doesn’t belong within the definition of the process.
Automation tools for both config and code based automation often offer functionality to add such things on top, separately from the automation itself.
What are the disadvantages of code based automation?
The disadvantage of course is that it is a lot more effort. All the complexity that config based automation hides from you is back on the users shoulders with code based automation.
So, what to do? How can I get the best of both worlds?
It is possible to combine both approaches to a best-of-breed approach called dynamic configuration management. This approach combines the use of a configuration database with code based automation to create ephemeral configurations which, depending on the setup, don’t ever need to end up in files.
What is dynamic configuration management?
Dynamic configuration management is the idea that configurations are ephemeral and therefore should be created dynamically, i.e. at the point of use, in the constellation required for that particular use. Dynamic configuration management is the epitome of automation that cleanly separates state and process.
How to use dynamic configuration management for automation
The basic elements needed to use dynamic configuration management for automation are:
- A configuration database which contains ephemeral configurations, as well as a configuration schema
- A good configuration schema which defines which configuration options are available
- A code based automation tool like Cloudomation Engine that is integrated with the configuration database, which allows to define the logic to get from current state to desired state as defined in the configuration
The user then once again only needs to define their desired state, supported by predefined options which are set in the configuration database. The code based automation then hurries along to create this desired state. Any customisations required can be done by tweaking the code.
The value of templates
A good automation system which implements dynamic configuration management will provide you with templates for everything, allowing you to get started quickly with a set of sensible defaults. You can then surgically customise if and when required. The main advantage: there will never be a case that you cannot implement. The main disadvantage: implementing complex customisation can involve a lot of effort.
Side note: why config based automation is common in infrastructure automation.
Templates can only be provided for specific use cases. The more specific the use case, the better the template. This is why config based automation is so common in infrastructure automation, but uncommon in other types of automation. Infrastructure automation is a fairly well-described use case. It is possible to create tools that implement logic to create a lot of desired states from config files, because there are standards and the number of options of desired states are manageable. This is not the case for most other automation use cases.
Conclusion
- Config based automation is simple to get started with, simple to use, but can be a pain to maintain for larger and more complex infrastructures.
- Code based automation is much more flexible, but requires more effort to set up (and can also be a pain to manage if the code is written poorly.)
- For both config and code based automation, there are many tools that help get a grip on the individual challenges such as maintaining and distributing config files, or managing code quality for process automation. However such additional tools also add complexity.
- Automation using dynamic configuration management combines the best of both approaches. It allows you to define every detail of how a desired state is created, while still enabling simple use by defining desired states as configurations, which can be maintained easily in a configuration database.
- For small(ish) infrastructure with low to medium complexity, config based automation is definitely the way to go.
- For larger or more complex infrastructure, code based approaches are often used because there is no alternative. For such constellations, using dynamic configuration management with code based automation can be hugely beneficial.
Side note
While the conceptual approach to automation is important, it is at least as important to choose tools that are mature and implement their conceptual approaches well. Using a poor configuration management tool with hand-written code will most likely not produce good results and be a pain to maintain. Using a mature config-based automation tool with a good ecosystem of supporting tools to manage it can produce very good results.
My recommendation: consider your requirements, pick an approach, and then go on the hunt for a good tool that implements this approach.
