Reference architectures for internal developer platforms (IDPs) have been all the rage lately, forming part of more than 20% of talks at the PlatformCon 2024 (as reported in the “State of platform engineering report vol 3” by Humanitec and Gitpod). They are useful in providing a common view on the pieces that can be part of an IDP. Here, I want to show where Cloudomation fits into these reference architectures.
Reference architectures and so called planes
The reference architecture popularised by platformengineering.org consists of five main components called “planes”:
- Developer Control Plane: All components through which developers interact with the platform. Typically includes GUIs / portals, source control, cloud development environments (CDEs), as well as configuration standards that developers maintain themselves (typically in their source code repository), such as ansible playbooks, helm charts, devfile.yaml etc.
- Integration & Delivery Plane: This is where all the automation happens. CI/CD pipeline tools, infrastructure automation, as well as other automation tools (e.g. platform orchestrators) are typically shown as part of this plane.
- Monitoring & Logging Plane: As the name suggests, this is where all observability tools are located.
- Security Plane: Secrets management, policy tools and other security tools.
- Resource Plane: Compute and storage.
Not all internal developer platforms necessarily have all five planes, and it doesn’t always make sense to divide an IDP into those five planes, e.g. when some of them are covered in the same tool or platform. However as a reference and starting point to talk about IDPs, this is a useful categorisation.
Where does Cloudomation fit into your IDP
As a pure Python framework, Cloudomation is a flexible solution that can provide all automation, orchestration and integration services for your internal developer platform. Cloudomation can be used as:
#1 Platform Orchestrator
This is the most common use case for our product Cloudomation Engine in an IDP. As platform orchestrator, it serves as the backend for your IDP. Engine is used to:
- Automate: With a focus on infrastructure and deployment automation, Cloudomation Engine can be used to natively automate resource and application deployment.
- Orchestrate: Tie together, manage and resolve configurations, and expose via a portal or custom REST APIs any existing automations you already have in place.
- Integrate: With its ETL features, Engine can fetch, validate, aggregate and expose via a custom API, via a portal, or via existing dashboard or monitoring tools data such as usage statistics, build statistics, log summaries and other insights collected from other tools in the IDP landscape.
#2 Cloud Development Environment (CDE)
We built Cloudomation DevStack to serve our own needs for one-click-to-deploy development environments for our engineers. It sits on top of Cloudomation Engine, using its automation capabilities to provision infrastructure and deploy software onto DevStack CDEs. It has a CLI and a lightweight web app to configure and manage CDEs. Read more about our unique approach to CDEs and how DevStack is different from other CDE solutions.
#3 CI/CD
With a particular focus on Continuous Deployment, Cloudomation Engine can be used to
- Natively build CI/CD pipelines from scratch, or migrate CI/CD pipelines from other tools to Cloudomation Engine (we offer migration as a service),
- Integrate existing tools into unified pipelines,
- Orchestrate existing pipelines,
- Extend existing pipelines.
#4 Configuration Management
With a built-in configuration and master data management system, Engine can be used to model your own unique configuration landscape. This typically consists of
- Several different configuration standards, which can be mapped, resolved and applied through Engine’s configuration management system,
- Other custom configuration information unique to your IDP, which can be modeled and enforced via Engine’s configuration management system.
#5 Portal
Engine forms are lightweight, json-schema-based forms which can serve as simple user interfaces to e.g. trigger a deployment, provide information about pipeline status, etc. Engine forms are useful for fast prototyping and as simple user interfaces for individual use cases, however they are not intended as a full portal solution. For an IDP, a dedicated portal (e.g. Backstage) makes sense, to which Cloudomation Engine can expose data and services, which developers then consume via the portal.
In addition, Cloudomation Engine has native integrations with:
- Git: All user content (Python scripts, configuration files, connectors etc.) created and maintained in Cloudomation Engine is versioned natively in git. Users can create and edit user content either via the Engine UI, or check out the git repository and make edits in their own IDE. On commit, their changes are synchronized with Engine.
- Hashicorp vault and devolutions secret manager: Native integration of these two secret managers allows users to access secrets in those secret stores directly in Python. Secrets are then fetched from the secret stores at runtime and never stored in Engine itself.
To integrate with and orchestrate other tools, Cloudomation Engine has connectors for
- AWS, Azure, Google Cloud and a REST connector that can be used with OpenStack (e.g. OpenShift),
- and ssh,
- as well as a lot of other connectors for tools, protocols and interfaces. See the full list of available connectors here in our documentation.
Summary
Cloudomation Engine is a Python-based platform orchestrator for your internal developer platform. You can build a minimum viable IDP with just Cloudomation, Backstage, and any cloud provider. However, Cloudomation really comes into its own in complex architectures with many components and complex configuration constellations, which it allows to manage sustainably.
A side note on platformengineering.org and PlatformCon
Platformengineering.org (as well as internaldeveloperplatform.org and possibly other innocuous .org domains) is owned and operated by PlatCo GmbH, which also develops and sells Humanitec as a product.
They are collaborating closely with Gitpod. Humanitec is primarily responsible for the content presented on those sites, with Gitpod as a significant contributor.
These .org sites pose as community websites and present content as “from the community, for the community”. The truth however is that this content is created, published and curated by PlatCo as a for-profit business and quite openly steers the content into a direction that always lists their products as well as Gitpod on top of all lists, uses them as best-practice examples, creates and popularises standards that are compatible with their products, and generally is used as a sales and marketing channel for their products.
PlatformCon is also organised by PlatCo / Humanitec, as is the platform weekly newsletter and the platformengineering slack channel.
I appreciate their support of the community and doubt that such a community would even exist in such vigor if it wasn’t for the active support of Humanitec (and Gitpod). It’s great that they are creating spaces for platform engineers to talk to each other, as well as information about platform engineering and internal developer platforms.
Nevertheless, I think it is important to be aware of who is behind this information and those community spaces, and to take their positioning of specific tools and standards (Humanitec products, Gitpod, Score) with a grain of salt.
