In this blogpost, I ain't describe in details what Pulumi is doing. I already talked about many times in previous blogpost but also in IT conference such as DevoxxFR:
Yeah I know it's in French, I'm sorry for non-French speaker1. Let me give you a bit of context: in this conference, we presented how Pulumi is working to allow people to us to use your favorite programming language to do some IaC and also how we can use it to transform this IaC as a real product with its own API and CLI. We called it Deployment as a Service or DaaS.
And that's why we're using it in our driver system2 for cwcloud:
If you want to learn more about it, we also detailed the DaaS concept in this tutorial.
So now we've said that this tool is more suitable for people who want to deliver their IaC as a Service, I'll also try to explain my point of vue of why this tool is better for almost everyone including people who enjoy using Terraform or IaC with declarative languages such as HCL (Hashicorp Configuration Language).
First of all, I think a declarative language such as HCL is kind of a bad compromise for people who ain't working the same way:
- classic system administrators who wants to only configure and avoid implementing any kind of logic
- SRE (System Reliability Engineer) or Platform Engineer who wants to use a turing complete programing language and be able to implement business logic in their IaC
The usual way to solve this is to use a configuration language such as YAML which is easy to read but also templatisable using an engine such as jinja2 (used by Ansible), or go template (used by helm).
However Hashicorp tried to reunite both needs with a single language for all its products including Terraform and it leads to something nobody likes very much, neither the developers nor the system administrators if you want my honest opinion3.
Here's an example of service we'd like to enable or disable with a enable_my_service flag and also manage high availability4 with another high_availability flag:
resource "aws_instance" "my_service" {
count = (var.enable_my_service == true ? (var.high_availability == true ? 3 : 1) : 0)
ami = data.aws_ami.ubuntu.id
instance_type = "t2.micro"
subnet_id = aws_subnet.subnet_public.id
tags = merge(local.common_tags)
}
You find this ugly? Wait for this: sometimes when the terraform provider isn't supporting an endpoint from the IaaS API, you have to use an "external" datasource.
Few years ago, that's something I had to do in order to get a PCS (Private Service Connect)5 id from GCP (Google Cloud Platform) and inject-it in the elastic provider to make the connection:
data "external" "get_psc_id" {
program = ["bash", "${path.module}/get_psc_id.sh", var.region, var.gcp_host_project, google_compute_forwarding_rule.psc_consumer[0].name]
count = (var.enable_psc == true ? 1 : 0)
}
resource "ec_deployment_traffic_filter" "traffic_filter" {
name = "${var.stage}-${var.project}-${var.region}-filter"
region = "${var.region_prefix}-${var.region}"
type = "gcp_private_service_connect_endpoint"
count = (var.enable_psc == true ? 1 : 0)
rule {
source = data.external.get_psc_id[0].result.pscConnectionId
}
}
And the program invoked by the external datasource has to be written in another programing language, such as bash in this example:
#!/usr/env/bin bash
set -eu
region="$1"
projet="$2"
name="$3"
jsonOutput="$(gcloud --project="$projet" compute forwarding-rules describe "$name" --format json --region "$region")"
pscId="$(echo "$jsonOutput"|jq -r .pscConnectionId)"
pscIp="$(echo "$jsonOutput"|jq -r .IPAddress)"
echo "{\"pscConnectionId\": \"${pscId}\", \"IPAddress\": \"${pscIp}\"}"
Using Pulumi, I'd be able to parse the output of the gcloud directly in my Python or Go code.
Having a tool reuniting multiple needs might be a good thing but in this case, I'd prefer when this tool is providing several languages. And that's exactly what Pulumi offers.
Indeed you can use Pulumi with your favorite programming language but there's also a simple declarative YAML interface available. Here's what it looks like (example from the pulumi blogpost):
name: yamldemo
runtime: yaml
resources:
bucket:
type: aws:s3:Bucket
properties:
website:
indexDocument: index.html
index.html:
type: aws:s3:BucketObject
properties:
bucket: ${bucket.id}
content: <h1>Hello, world!</h1>
contentType: text/html
acl: public-read
outputs:
url: http://${bucket.websiteEndpoint}
In my opinion it's a better approach to continue to answer everyone's needs: choose the language you like to work with, including the most known declarative language used for configuration which is YAML nowadays. And even if you don't like YAML, it's pretty easy to produce a YAML file from another format (way more easy than producing code from templates) ;p
That been said, it's hard to catch up after years or decades of cloud players or vendors interfacing their IaaS (Infrastructure as a Service) or SaaS (Software as a Service) with public Terraform providers available on public registries. However at some point it happen: we can remember the hegemony of Puppet before Ansible during years.
In the case of Pulumi, despite the fact they've already done the job for most of the big cloud players, they also made very smart moves like:
- providing tools to convert a terraform provider into pulumi SDK in every supported languages
- you can also include terraform as dependancy directly in your Pulumi code
- you can convert the state of resources earlier created with terraform into pulumi states
This makes it so easy to mitigate until all the providers understand that Pulumi is the way!
Pulumi isn't only an alternative for Terraform and IaC world with classical IaaS resources but it's shaking up the Kubernetes (K8S) world beeing the first solid alternative to helm.
Despite I like it very much and despite the fact that the go template doesn't bothers me at all, I must admin that it's hate by a lot of people who prefer to use kustomize despite the duplication it generate or even prefer to handle K8S manifests themselve with some piece of code...
That's why Pulumi is providing a Kubernetes package and I think it could be the right call for those people because they'll be able to implement deployment logic using the programing language they like instead of a templating engine they despise.
And what is beautiful is that you can also include and re-use public helmcharts coming from public regisries exactly the same way you can re-use Terraform provider inside your Pulumi code.
Here's an example of invoking with values the nginx-ingress helmchart inside a Pulumi Python's code (example from the official documentation):
from pulumi_kubernetes.helm.v3 import Chart, ChartOpts, FetchOpts
nginx_ingress = Chart(
"nginx-ingress",
ChartOpts(
chart="nginx-ingress",
version="1.24.4",
fetch_opts=FetchOpts(
repo="https://charts.helm.sh/stable",
),
values={
"controller": {
"metrics": {
"enabled": True,
},
},
},
),
)
Amazing isn't it?
To conclude, we can see how Pulumi smartly meets a large number of needs in the IaC world: people who prefer to configure, those who prefer to develop, people working with K8S or classical IaaS resources...
Like I said multiple times in my previous blogposts for PaaS (Platform as a Service) or frameworks and it includes IaC tools as well: interoperability, agnosticity and polyglotism are keys to success.
- I plan to have this talk in future english speaking events, maybe a Pulumi's official meetup, stay tuned :p↩
- At the time of writing we succeed to develop a driver, using the available pulumi modules, for the following cloud providers: AWS, GCP, Azure, Scaleway, OVH and Cloudflare↩
- Of course you'll find people who says they like Terraform and HCL but keep in mind that Terraform is 10 years old now and the challengers like Pulumi or Crossplane aren't that old. So it make sens that lot of them could have developed some kind of "digital Stockholm Syndrome" because they succeed to use it for years↩
- Let's assume High Availability means deploying three nodes of this service↩
- It's a way to establish private connections between your VPC and external services hosted somewhere else like elastic cloud in this example. More information here.↩