Part 2: Improving the Workflows

This is Part 2 of a series illustrating how to write Kratix Promises.

👈🏾 Previous: Writing your first Promise
👉🏾 Next: Accessing secrets and storing state

In the previous tutorial, you wrote your first Promise, delivering Apps-as-a-Service. In this section, you will dive deeper into Promise Workflows.

You will:

Read about Workflow design patterns
Learn how to test your workflows

Workflow design patterns

To recap, a Kratix Promise is configured with a collection of Workflows that can be triggered at different stages of the Promise or Resource lifecycle. In the previous section, you defined a Workflow for the resource.configure action. In that Workflow, you defined a collection of Kubernetes Resources that needed to be created to fulfill the user's request.

However, there's virtually no limitation to what you can do in a Workflow. You can, for example, download software, run imperative commands, wait for manual approvals and more. Pipeline stages are meant to be reused across different Promises, so you can define a Workflow once and reuse it across multiple Promises.

To make that possible, it's important to follow some design patterns when writing Workflows. In this section, you will learn about some of these design patterns.

Reusability

Workflows are a great place to validate and enforce common requirements. For example, if you write a stage that can check for necessary compliance requirements, that stage can be used by all applicable Pipelines. In addition, you can write stages to check for excess costs, labelling conventions, security risks, and more.

While most Workflows will have at least one stage with logic unique to that Promise, building the Kratix Pipeline stages with reusability in mind is a great way to make your platform extensible.

Idempotency

An idempotent Workflow guarantees that running the same command multiple times will result in the same outcome. This is an important feature because they will be auto-reconciled on an ongoing basis.

Kubernetes controllers reconcile their objects in three scenarios:

Object change
Controller restart
Default cadence

This means that yes, the Workflow will run on every request for a Resource. But also, it will run any time the controller is reset, which by default is every 10 hours.

This means you will need to write your Workflows to make sure that re-running them will not result in any adverse side effects. This is especially important when your Workflow is interacting with external systems. In the next tutorials, you will learn some strategies to make your Workflows idempotent.

🤔 Wondering when to use Workflows versus creating a new Promise?

Platform design requires thinking about how to divide platform offerings into right sized Promises and evaluating options for reusability and composability.

Each Promise is the encapsulation of something as-a-Service. But that doesn't mean that all platform users will want or need all types of Promises. It can be extremely helpful to create lower level Promises for services that are composed into a number of higher level offerings. For example, a Kubernetes Promise may never be something requested by an application developer, but it may be that a number of software Promises like "environment", or "data store" depend on a Kubernetes cluster that can be provisioned using a Promise.

Promises are not the only way to create reusable components when designing your platform with Kratix. You can also create reusable Pipeline stages that can be run in a number of different Promise Workflows. For example, you may want to add default labels to certain types of resources. You can create a Pipeline stage which evaluates the resources set to be declared at the end of the Workflow and apply consistent labelling before writing.

Since both Promises and Workflows can be reused, you may wonder when to use each. The best rule of thumb is to ask if you are describing a noun or a verb.

Nouns are most easily described as things. A database is a thing, so is a cluster, or an application, or any number of software offerings your platform may support. If you are trying to provide something as-a-Service you should be thinking about creating a Promise.

Verbs can be described as actions. Labelling, notifying, or scanning can all be actions you may want to take rather than things you want to create. These actions can often be made across multiple things, e.g. you may want to label both databases and queues. When you are trying to take action to fulfil a cross-cutting concern, this is most suited to a Workflow step.

Like all rules of thumb, this should be treated as a guide. When it comes to system design it is important that it works for your context and the Syntasso team is happy to work with you as you approach these discussions as a team.

Testability

One of the advantages of the Kratix workflow model is that it makes it easy to test your workflows. You can test your workflows in isolation, without having to deploy them to a cluster. This makes it easy to test your workflows as you are developing them.

The next section goes into more detail about how to test your workflows.

Test driving your workflows

In the previous tutorial you created a simple workflow that generated the Kubernetes resources needed to fulfill the user's request. You tested it by installing the Promise and then sending a resource request to the Platform.

A better way to test your workflows is to just execute the container in your local environment. It's relatively easy to do this, and it will save you a lot of time as you are developing your workflows.

Prepare your environment

When Kratix starts your workflow, it will mount the following three volumes:

/kratix/input: Kratix will add the user's request into this directory as object.yaml.
/kratix/output: the files in this directory will be scheduled to a matching Destination.
/kratix/metadata: a directory where you can write documents with special meaning to Kratix; you will cover this in the next tutorials.

That means that, to test it locally, you will need to provide the three volumes to your Docker command. Note that you will also need to write the object.yaml containing an example request in the input directory.

Still in the app-promise directory, run:

mkdir -p test/{input,output,metadata}
touch test/input/object.yaml

Later, when you run docker run, you will mount the three directories onto the container. But first, you need to create the object.yaml file in the input directory. Create a file called test/input/object.yaml with the following content:

app-promise/test/input/object.yaml
apiVersion: workshop.kratix.io/v1
kind: App
metadata:
  name: my-app
  namespace: default
spec:
  image: example/image:v1.0.0
  service:
    port: 9000

With that in place, you can go ahead and test your pipeline! Since the next tutorials will ask you to rebuild and retest the image, it's a good idea to write a simple helper script that you can re-run in the future. Run the following:

mkdir -p scripts

cat << 'EOF' > scripts/build-pipeline
#!/usr/bin/env bash

set -eu -o pipefail

testdir=$(cd "$(dirname "$0")"/../test; pwd)

docker build --tag kratix-workshop/app-pipeline-image:v1.0.0 workflows/resource/configure/mypipeline/kratix-workshop-app-pipeline-image
kind load docker-image kratix-workshop/app-pipeline-image:v1.0.0 --name platform
EOF

cat <<'EOF' > scripts/test-pipeline
#!/usr/bin/env bash

scriptsdir=$(cd "$(dirname "$0")"; pwd)
testdir=$(cd "$(dirname "$0")"/../test; pwd)
inputDir="$testdir/input"
outputDir="$testdir/output"
metadataDir="$testdir/metadata"

$scriptsdir/build-pipeline
rm -rf $outputDir/*

command=${1:-"resource-configure"}

docker run \
    --rm \
    --volume ~/.kube:/root/.kube \
    --network=host \
    --volume ${outputDir}:/kratix/output \
    --volume ${inputDir}:/kratix/input \
    --volume ${metadataDir}:/kratix/metadata \
    --env MINIO_USER=minioadmin \
    --env MINIO_PASSWORD=minioadmin \
    --env MINIO_ENDPOINT=localhost:31337 \
    kratix-workshop/app-pipeline-image:v1.0.0 sh -c "$command"
EOF

chmod +x scripts/*

warning

If you are running the workshop on a Mac, you may need to update the MINIO_ENDPOINT above to host.docker.internal:31337.

These scripts do the following:

build-pipeline codifies the image building and loads the container image on the kind cluster.
test-pipeline calls build-pipeline and also runs the image, allowing you to verify the created files in the test/output directory.

At this stage, your directory structure should look like this:

📂 app-promise
├── README.md
├── dependencies
│   └── dependencies.yaml
├── example-resource.yaml
├── promise.yaml
├── scripts
│   ├── build-pipeline
│   └── test-pipeline
├── test
│   ├── input
│   │   └── object.yaml
│   ├── metadata
│   └── output
└── workflows
    └── resource
        └── configure
            └── mypipeline
                └── kratix-workshop-app-pipeline-image
                    ├── Dockerfile
                    ├── resources
                    └── scripts
                        └── resource-configure

Run the tests

To execute the test, run the script with the following command:

./scripts/test-pipeline resource-configure

Which should build and run the image. Once the execution completes, verify the test/output directory. You should see the following files:

📂 app-promise
├── ...
└── test
    ├── input
    │   └── object.yaml
    └── output
        ├── deployment.yaml
        ├── ingress.yaml
        └── service.yaml

You can take a look at the files and verify their contents. You can see where the user's inputs are being used, and how that is reflected in the created services. You can poke around and make changes to both the input and to the workflow, and see how the output reflects this.

The ability to treat images as independent pieces of software that can have their own development lifecycle (fully testable, easy to execute locally, release independent) allows platform teams to move faster, sharing and reusing images across their Promises.

Next, you will explore how to access secrets and store state from within a workflow.

🎉 Congratulations!

You iterated over the Promise and added a Promise Workflow! Congratulations 🎉

To recap what you achieved:

✅ You got a deeper understanding of the Workflows.
✅ You tested your Resource Workflow.

👉🏾 Let's explore accessing secrets and storing state in workflows.

Workflow design patterns​

Reusability​

Idempotency​

Testability​

Test driving your workflows​

Prepare your environment​

Run the tests​

🎉 Congratulations!​