Hey! We made a thing!

At KZN Group, we are big fan of Jsonnet. Jsonnet is a small (functional) programming language developed as a simple extension of Json. Basically Jsonnet is Json plus variables, functions and few syntax improvements. You’re invited to check its website, but, as a taster, let me copy an example from it:

This:

local Person(name='Alice') = {
  name: name,
  welcome: 'Hello ' + name + '!',
};
{
  person1: Person(),
  person2: Person('Bob'),
}

becomes this:

{
  "person1": {
    "name": "Alice",
    "welcome": "Hello Alice!"
  },
  "person2": {
    "name": "Bob",
    "welcome": "Hello Bob!"
  }
}

You might think, Jsonnet can be helpful if you had to generate large Json documents, and, if you are familiar with AWS CloudFormation, you might be scratching your head already.

Indeed, since the beginning of this year, Jsonnet has been our card up our sleeve to efficiently compose large CloudFormation templates for our customers. If you want to know more about CloudFormation and Jsonnet, here is a presentation I recently gave at the Perth AWS User Group.

Jsonnet as a CloudFormation Macro

When we read about CloudFormation Macros, things got even more exciting.

Previously, our workflow consisted in creating a CloudFormation template in Jsonnet and render it to Json just before creating a stack. Now, exploiting CloudFormation macros, we can delegate this preprocessing step to CloudFormation itself. Moreover we can start with a YAML template and add little bits of Jsonnet here.

As an impromptu experiment, we created cf-macro-jsonnet, admittedly inspired by Jay McConnell’s PyPlate.

cf-macro-jsonnet is a CloudFormation macro that allows you to embed arbitrary Jsonnet code into your CloudFormation templates.

Here is a simple example. There are few things in life that bother me more that having to list tags in a CloudFormation resource as an arrary of keys and values 😂. So I can do this instead:

AWSTemplateFormatVersion: "2010-09-09"
Transform: [Jsonnet]
Resources:
  S3Bucket:
    Type: "AWS::S3::Bucket"
    Properties:
      Tags: |
        #!jsonnet
        local tags = {
          Project: 'take-over-the-world',
          Stage: 'testing'
        };
        [{ Key: k, Value: tags[k] } for k in std.objectFields(tags)]

The macro will traverse the template looking for strings that start with #!jsonnet. Such strings will be evaluated as Jsonnet code and the result will be put back in place of the original string. So the above will be rendered (and executed!) as I had written this.

AWSTemplateFormatVersion: "2010-09-09"
Resources:
  S3Bucket:
    Type: "AWS::S3::Bucket"
    Properties:
      Tags:
        - Key: Project
          Value: take-over-the-world
        - Key: Stage
          Value: testing

This has good potential but if you look carefully we actually had to write two extra lines in our Jsonnet version.

So we went a step further.

Jsonnet suports importing code from a separate file. This allows us to create a library of reusable snippets. Following from the previous example, we can create a file utils.jsonnet with content:

{
  toEntries(obj): [{ Key: k, Value: obj[k] } for k in std.objectFields(obj)],
}

and call it this way:

local utils = import "utils.jsonnet";
utils.toEntries({
  Project: 'take-over-the-world',
  Stage: 'testing'
})

cf-macro-jsonnet hooks into this mechanism: all you have to do is zip up your Jsonnet library and upload it to S3. Then we can use the library from a templates.

AWSTemplateFormatVersion: "2010-09-09"
Transform: [Jsonnet]
JsonnetLibraryUri: s3://path/to/zip/file/above
Resources:
  S3Bucket:
    Type: "AWS::S3::Bucket"
    Properties:
      Tags: |
        #!jsonnet
        local utils = import "utils.jsonnet";
        utils.toEntries({
          Project: 'take-over-the-world',
          Stage: 'testing'
        })

Which still has the same number of lines but it’s way cooler 😂. Jokes aside, there is no limit to the amount of complexity you can hide inside library function, so this is cleary an improvement. I can think of few ways to add things to the scope, avoding the explicit local binding; but I need to mull over it a bit longer.

Ideas

To give an idea of how we are using Jsonnet to write CloudFormation template more efficiently, let me show an example. Writing a IAM role can be tedious:

{
  "Type": "AWS::IAM::Role",
    "Properties": {
      "AssumeRolePolicyDocument": {
        "Statement": [ {
          "Effect": "Allow",
          "Principal": {
            "Service": [ "lambda.amazonaws.com" ]
          },
          "Action": [ "sts:AssumeRole" ]
        } ]
      },
      "Path": "/",
      "Policies": [ {
        "PolicyName": "InvokePolicy",
        "PolicyDocument": {
          "Statement": [ {
            "Effect": "Allow",
            "Action": [
              "lambda:InvokeFunction"
            ],
            "Resource": [
              "*"
            ]
          } ]
        }
      } ]
    }
}

How about the following

#!jsonnet
local utils = import 'utils.jsonnet';
utils.assumeRole('lambda.amazonaws.com', [{
  PolicyName: 'InvokePolicy',
  PolicyDocument: {
    Statement: [{
      Effect: 'Allow',
      Action: [
        'lambda:InvokeFunction',
      ],
      Resource: [
        '*',
      ],
    }],
  },
}])

or even

#!jsonnet
local utils = import 'utils.jsonnet';
utils.assumeRole('lambda.amazonaws.com', [{
  'Allow:*': 'lambda:InvokeFunction',
}])

where we write togheter the policy’s effect and resource. Of course this can’t express all possible policies but it does cover the most common cases.

The implementation of assumeRole is left to the reader, pull-requests are welcome :)

Thanks to Simon T, Eric H, and Ric H. from AWS who encouraged me to jot down this blog post.

For questions, suggestions or good jokes, don’t hesitate to ping me on Twitter or by opening an issue on the cf-macro-jsonnet repository.