Devfile & devcontainer vs. Dockerfile & Docker-Compose

Devfile.yml and devcontainer.json are both standards to describe development containers. Different implementations exist that take these standards and deploy a container based on these files.

When I first heard about devfile, my first thought was: What’s the difference to Dockerfile? A Dockerfile also describes a container, no matter whether it is a production, development or any other container. I also wondered: Why do I need a specific standard for development environments? Isn’t a container a container, and what’s inside it shouldn’t matter to the standard that describes it?

The devcontainers documentation tries to answer this question by explaining that development containers typically contain additional packages and tools that are not part of a production environment. However I was still not clear why this requires a separate standard. Couldn’t I just use Dockerfiles to describe different containers for development and production?

The answer is: yes, you can. But devcontainer and devfile still provide additional functionality that goes beyond what a Dockerfile on its own can do.

Before comparing them, I will briefly describe what Dockerfile, Docker-compose, devfile and devcontainer do.

Dockerfile

Dockerfiles contain instructions to build a Docker container image. One Dockerfile describes one single container. Docker can build a Docker container image from a Dockerfile. For example, a Dockerfile specifies

which operating system should be used in the container,
it can add files and packages to the container,
it can specify shell commands that should be run in the container etc.

The Dockerfile contains all information needed to build a container from first principles: specifying everything that the container should contain.

Docker-compose

Docker-compose on the other hand allows you to describe multi-container applications. In a Docker-compose.yaml, you can reference several Docker containers and how they should interact with each other. For example, in a Docker-compose file you can reference a frontend, a backend and a database container, define a network for the containers to communicate with each other, map ports, inject secrets and attach storage volumes to the containers. The Docker-compose file can reference container images in a container registry, or reference Dockerfiles to build the containers from scratch.

Devfile & devcontainer

Devfile and devcontainer provide yet another layer on top, which allows you to describe a full development environment. That includes referencing container images (from a registry, or Dockerfiles or Docker-compose files) as well as other components that make up a full development environment, such as IDE plugins, source code repositories, and commands that should be run in the development environment after its creation.
The devfile and devcontainer standards can be used with different tools which interpret and execute them. The majority of these tools are IDEs. Find a list of compatible tools in their respective documentation:
Devcontainer: https://containers.dev/supporting
Devfile: https://devfile.io/docs/2.2.2/innerloop-vs-outerloop#support-list-of-developer-tools

Devfile and devcontainer contain:

“Interpreter-specific” configuration: for example, VS code extensions that are installed when a devcontainer.json is interpreted (i.e. executed) by VS code, or the directory that contains the source code that should be opened and mapped by the IDE.
Container customisation: port mappings (if those are not already specified in a referenced docker-compose file) and commands to run inside the container e.g. to start services,
Script(s) to run after deployment of the containers, e.g. a CLI that guides you through configuration of the application (e.g. to create a user) or a script that creates and loads an ssl certificate.

For example using a devfile or a devcontainer file could look like this:

You clone a source code repository that contains a devfile or a devcontainer.json.
You open the devfile or devcontainer.json in an IDE that supports these standards.
The IDE asks you if you want to execute the configuration. If you choose to do so, it will do what is specified in the configuration. For example, it could:
Pull images from a registry or create images from dockerfiles /docker-compose file in the repository,
Run the commands from the devfile or devcontainer.json in the container,
Run a post-installation script,
Install IDE plugins,
Open the source code repository in the IDE.

There are requirements that need to be fulfilled for this to work. For example, Docker needs to be installed on your system and the IDE needs to be configured correctly so that it can run Docker commands.

The containers that are deployed in this process typically contain the application that you’d want to work on, with all dependencies that it requires. This is done by Docker and requires properly configured Docker containers.

To summarise, what the devfile and devcontainer.json standards provide is:

Convenience for Docker functionality: in the config file, the Docker commands are specified and executed by the IDE,
Convenience for the execution of other configuration steps such as running post-installation scripts, and
IDE configuration.

Differences

Contrary to Dockerfiles, devfile and devcontainer don’t build a container from first principles, but add an additional layer on top of existing Docker container images.
Contrary to Docker-compose, devfile and devcontainer do not specify how several containers should work together.

Similarities

Some functionality is however available in some or all of these standards.
All of them allow to specify commands that should be run inside the container at different stages of its creation or deployment. This is sensible and even necessary to allow proper separation of concern and reusability of configuration.

A dockerfile that describes how a single container is built can be reused for many different purposes, for example to run an application in development, test and production environments. The commands that are specified in the dockerfile should be generic and serve only the purpose of defining all the contents of the container.

The docker-compose file describes how different containers should work together to form a multi-component application. Therefore, the docker-compose file has awareness of other containers that the single container lacks. Commands specified in the docker-compose file can execute commands that are only possible once the other containers exist, or that only make sense with other containers present – for example, attaching the containers to the same network.

In a development environment, other commands might make sense that are development-specific, such as preloading an application with test data or injecting access keys to a development cluster.

Port mappings can be specified in both docker-compose as well as devfile and devcontainer files. If a docker-compose file is referenced in a devfile or devcontainer file, then the port mappings can be contained in the docker-compose file. However, devfile and devcontainer allow reference to finished container images as well as dockerfiles, which don’t contain port mappings.

What’s missing

One thing that all these standards lack is specification of the underlying infrastructure on which to run the containers or development environments. (Though both Devcontainer and Devfile allow to specify some aspects of this, e.g. minimum host requirements (Devcontainer) or how a deployment to Kubernetes should look like (Devfile).)

This goes back to the topic of separation of concerns: there are separate tools out there that are specialised on infrastructure automation, which can be used in combination with the described standards to automate the creation of full environments. Typically, however, the underlying infrastructure is the developer’s laptop.

Conclusion

Looking at the functionality provided by devfile, devcontainer, dockerfile and docker-compose, each has a valid place in the configuration and automation toolstack for local deployment of software as part of a development environment.

The worrying impression that I am left with, however, is that all this is incredibly complicated. There are several layers of complexity, each of which warrants its own standard. Apart from the effort involved in understanding and applying all of these configuration layers and how they interact, the distribution of functionality into many different, partially overlapping configuration standards creates a large number of potential points of failure. The fact that configuration is distributed across several individual config files also makes it difficult to pinpoint and correct issues when they occur.

This is exacerbated by the ambition of devfile and devcontainer to serve as broad standards, which requires them to support a large number of possible deployments, leading to increasing complexity in their schemata.

Already, devcontainer supports additional sub-files for so-called features, which further extend devcontainer functionality. On the one hand, this looks like feature-creep which is common with popular tools which accumulate bloat as they are expanded to support more and more edge cases. On the other hand, confronted with the myriad of different configuration standards that already exist, some of which cater to a very narrow set of use cases (like devfile and devcontainer), I wonder if it might not be an improvement if some of them broadened their scope so that at least I don’t have to add yet another configuration standard to an already complex stack.

How to solve this?

What is needed to deal with this increasing complexity is a layer of abstraction that allows developers to consume deployments as a service without having to worry about the required configuration. This is what we are providing with Cloudomation DevStack, our platform for remote development environments. It allows a specialised team of DevOps or platform engineers to provide development environments as a service to developers. Developers themselves are no longer required to handle the increasingly complex stack of interdependent configurations just to get their local development environments running.