jk diary: packaging a jk script with kpt

In my previous outing with kpt, I managed to make a JavaScript program into a container image that could be used with kpt fn to create some Kubernetes configuration. An obvious question, having reached that summit, is

Can you use that image with the other bits of kpt?

To be able to answer in the affirmative, I need to demonstrate:

• making a package someone can import with kpt pkg
• giving that package some settings for use with kpt cfg

A demonstration is in https://github.com/squaremo/kpt-generator-demo – here I’ll explain some of the process of getting there.

Making a package

The easy bit is this:

kpt pkg init . --name kpt-demo

That creates a Kptfile in the current directory and gives it the name kpt-demo. (The more economical mode of use, just kpt pkg init DIR, is for creating a package from outside the directory containing the goodies.)

The Kptfile, as this point, looks like this:

apiVersion: kpt.dev/v1alpha1
kind: Kptfile
metadata:
  name: kpt-demo
packageMetadata:
  shortDescription: Demo of generating resources with kpt

Pretty self-explanatory so far. I’m not convinced by this fashion of co-opting Kubernetes' TypeMeta and ObjectMeta structures (the apiVersion, kind, and metadata fields) for config files that aren’t intended for the Kubernetes API. Kustomize does this too, and I think it just confuses and complicates matters.

Moving on, what’s in the package?

What lies within

I borrowed the technology developed in the last post for building a container image; it’s in image/. The kpt bits assume the image is available in the local Docker with the name generate – e.g., by building it with the following:

docker build -t generate ./image

The script generate.js in there went through a few revisions. At first I tried to make it work in different modes:

• kpt fn run . scoops up all the resources found within ., then finds any resources that define themselves as functions (with the config.kubernetes.io/function annotation, and runs them;
• kpt fn run . --image=generate -- ... scoops up the resources found within ., and runs the image generate on them (with any parameters supplied after a --)

Both of these will replace the files in . with those that come out the other side of the image (and remove any files that weren’t in the output).

Clearly the idea is that functions go through and modify things in place, and otherwise repeat back whatever they got as input. In my case, though, I want to assert the resources in the package, rather than transform them. If the config is part of the input, it needs to be part of the output, otherwise it will be erased, and running the same thing again won’t necessarily get the same result.

It’s less fiddly if the function config lives off to one side in fn/ – and this is more suitable for kpt cfg, as you’ll see.

The second revision of the script does not take into account the function config, and just generates the desired resources. It doesn’t expect, or output, the resource that’s used as the functionConfig. To keep the config and the output separate, the output goes in instance/, and the invocation to generate it is now:

kpt fn run ./instance --fn-path=./fn

Parameterising the generation step

The script can be given a functionConfig object (part of the kpt fn protocol), from which it gets values for namespace and image.

Since the functionConfig can be a resource itself, its fields can be set by kpt cfg, though you can only set scalar values (numbers, strings and booleans), while a functionConfig could have composite values.

Creating a setter is simple:

kpt cfg create-setter . namespace default

This does two things: it creates a record of the setter in the Kptfile, and it marks all the fields it can find with that value, as being set by the setter. In my case, that includes the generated files, which is not what I want – it’s only the functionConfig that matters.

Rerunning the generation step erases the marks in the generated files. Using kpt cfg with the functionConfig relies on that file not being amongst the generated files, for that reason – it would lose the setter marking, which is encoded in a comment.

Using the package in a configuration

With the setters set, it’s possible to import the package into another configuration and customise it there.

mkdir /tmp/newconfig
cd /tmp/newconfig
git init
kpt pkg get https://github.com/squaremo/kpt-generator-demo.git helloworld
kpt cfg set helloworld namespace hello
kpt fn run helloworld/instance --fn-path helloworld/fn
kpt cfg tree helloworld/instance
# ...

There’s an extra kpt fn step after setting the namespace, because the files must be regenerated.

Where this gets us

The demo repo shows how to package a JavaScript program into a container image, then use that image with kpt fn to generate configuration. The config used to specify the function is kept off to one side, so it’s not part of the generated files, and can be altered with kpt cfg.

It seems reasonable to assume that you could also containerise Helm charts, or indeed other programs, and use them in a similar way. To me this is superior to just splatting the (e.g.) Helm chart into YAMLs and making that your kpt package, as suggested in the kpt docs. If the configuration in the chart can just be rendered out as YAMLs with any or no parameters and be a useful package, why is it in a chart?

I like the way kpt gives you tooling to manage packages of plain YAMLs, with clever updating. I also like the idea of using programs to generate configuration, since plain YAMLs with the ability to set some field values is totally inadequate as a reusable package. Lots of things are easier with concrete values, but: abstractions have power!