jk diary: using jk with kpt

Recently Google open-sourced their project kpt, which is for managing Kubernetes configurations. It’s a well thought-through set of tools that work in sympathy with each other, with a minimal bit of protocol (that is, things that you as the user need to keep in order) so they can interact.

Where does jk fit in with kpt?

One of the tools in kpt is kpt fn, which is a way to run containers to transform the files in a directory. There are three subcommands:

• kpt fn source – generate Kubernetes config;
• kpt fn run – run a container to transform or inspect config;
• kpt fn sink – process config.

You can see already that kpt fn is something you might want to use with jk – let’s try it!

Can jk be used with kpt fn source

My first idea is that jk could be used as a source of configuration, i.e, with kpt fn source.

There is a specification for container images you can use with kpt fn. Notice that it’s actually part of the Kustomize documentation – kpt fn is borrowed from Kustomize.

The specification amounts to this: you read a ResourceList document from stdin, which might come with functionConfig; and, you print a ResourceList document to stdout.

My basic plan here is to make a container image that will output what kpt fn expects. Here’s an example from the function catalogue, which expands a Helm chart into the format expected by kpt fn.

It’s a bit mysterious how the container gets access to files, i.e., the chart, in the host filesystem – I mean, yes it’s because there’s a mount into the container, but what is mounted where?

Looking at the end to end tests for that helm-template image, I see it doesn’t actually work with kpt fn as I expected. This seems to be for a few reasons:

• kpt fn source doesn’t let you supply a container image with a flag, despite there being “source” functions in the catalogue;
• there’s no way to mount a volume when running a kpt fn command, so you can’t make arbitrary files (e.g., the Helm chart) available to the function. This might appear in a release in the near future though;
• the example doesn’t examine the functionConfig given in the spec (i.e. doesn’t follow the protocol); it just expects the arguments to be supplied to its script – so if you try to run it with kpt run, you just get the usage message.

Apparently the examples are running a little ahead of what’s actually supported in the tools.

However, I can work within these constraints, by including all the JavaScript code in the image, and using the functionConfig as parameters. But I’ll need some scaffolding.

Making a kpt fn runnable image

To recap: I wanted to make an image that could be used with kpt fn source, which would run a script in whichever directory. But:

• you can’t use kpt fn source that way; and,
• you don’t get access to files in the directory.

I can still use kpt fn run, and include the files of interest within the image. Then I can invoke it with something like:

$ kpt fn run . --image jk-generator-fn

Or even, where there are function definitions in the directory,

$ kpt run .

This situation is not terrible: if you were using jk to make resuable bits of configuration, you might do something like this anyway, building your packages into images, then referring to them (with some parameters) in your config repo.

Onwards. Here’s a simple script that generates a couple of Kubernetes resources, and puts them in a ResourceList so kpt fn will be happy:

// generate.js
import { core, apps } from '@jkcfg/kubernetes/api';
import { read, write, stdin, stdout, Format } from '@jkcfg/std';

class ResourceList {
  constructor(items) {
    this.items = items;
    this.kind = 'ResourceList';
    this.apiVersion = 'config.kubernetes.io/v1beta1';
  }
}

async function main() {
  const input = await read(stdin, { format: Format.YAML });

  const items = [
    new apps.v1.Deployment('deploy', {
    }),
    new core.v1.Service('srv', {
    }),
  ];
  const rl = new ResourceList([...items, ...(input) ? input.items : []]);
  write(rl, stdout, { format: Format.YAML });
}

main();

A couple of things to notice:

• it reads from stdin first, in case it got things piped to it
• it includes the piped-in resources in the output

It turns out these are crucial when using it with kpt fn run, because it will prune files that aren’t in the output. And I need at least one YAML file to be present, as you’ll see.

There’s a couple of dependencies for this script that will need to go in the image. The jk executable itself, and the library @jkcfg/kubernetes. Here’s a Dockefile that will download those as well as copy in the script:

FROM alpine:latest

WORKDIR /jk
COPY --from=jkcfg/kubernetes:0.6.2 /jk/modules .
ADD https://github.com/jkcfg/jk/releases/download/0.4.0/jk-linux-amd64 ./jk
RUN chmod a+x /jk/jk
COPY generate.js ./
ENTRYPOINT ["/jk/jk", "run"]
CMD ["./generate.js"]

I’ve based it on alpine simply so that I have chmod there to set the downloaded file to be executable. If there were a tarball I could expand, I wouldn’t need it.

@jkcfg/kubernetes is a library image, and keeps its code under /jk/modules/; to make it resolvable from the script, the contents of that directory get copied alongside, into /jk (reminder, COPY copies the contents of a directory, not the directory).

This will build the image:

$ docker built -t jkgen .

Let’s test it:

$ docker run --rm jkgen
apiVersion: config.kubernetes.io/v1beta1
items:
- apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: deploy
- apiVersion: v1
  kind: Service
  metadata:
    name: srv
kind: ResourceList

Looks reasonable. What about running it with kpt fn run?

$ kpt fn run . --image jkgen --dry-run

Um, no output. It turns out that if there’s no YAML files, kpt fn decides there’s nothing to do. Which makes some sense for kpt fn run, perhaps less so for kpt fn source, at least according to my expectations.

I can kill two birds with one stone here, though: you can specify a function with a YAML file, and this will also give kpt fn a resource so there’s something to process.

apiVersion: v1
kind: ConfigMap
metadata:
  annotations:
    config.k8s.io/function: |
      container:
        image: jkgen      
    config.kubernetes.io/local-config: "true"
  name: jkgen
data:
  app: foobar

Now I have all the ingredients:

• an image that obeys the kpt fn protocol;
• a declarative specification for calling the image as a function;
• a YAML that kpt fn run can process.

$ kpt fn run . --dry-run
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deploy
  annotations:
    config.kubernetes.io/path: 'deployment_deploy.yaml'
---
apiVersion: v1
kind: Service
metadata:
  name: srv
  annotations:
    config.kubernetes.io/path: 'service_srv.yaml'
---
apiVersion: v1
data:
  name: foobar
kind: ConfigMap
metadata:
  annotations:
    config.k8s.io/function: |
      container:
        image: jkgen
    config.kubernetes.io/local-config: "true"
    config.kubernetes.io/path: jkgen.yaml
  name: jkgen

Success!

Where to now

To summarise where I got to: I wrote a script for jk and put it in an image, and could use that with kpt fn run, so long as I played by some rules:

• you have to supply at least one YAML, since kpt fn run is for transforming things;
• you have to be careful not to remove things that were given to you as input, since kpt fn run will delete things that don’t appear.

There is a little friction in how I’m using kpt fn run; but at the same time, I don’t think the kpt developers are quite finished with e.g., how kpt fn source works, judging by the examples they’ve lined up, so maybe that awkwardness will be ironed out.

I think there is a lot of promise here, and working well with kpt is an appealing aim. There are some things jk could do in that direction:

• Have a @jkcfg/kubernetes/kpt module, for dealing with the kpt fn protocol;
• Make building function images from jk scripts easy (the kpt function SDK does a really nice job of this)
• further experimentation with using jk for e.g., blueprints (a part of kpt that seems speculative, at present)