AI-Powered Kubernetes Testing Using GPTScript and Testkube

I’ve been using GPTScript extensively to build agents to perform specific tasks and improve my productivity for a few months. My previous post discussed AI-powered DevOps and how I built an agent to work with GCP and Kubernetes.

With microservices and distributed applications, testing has become more complex than ever. From creating and scaling tests across distributed systems to keeping up with the pace of deployments, managing tests efficiently is difficult.

In this post, I focus on testing and see how we use GPTScript to work with Testkube, a Kubernetes-based testing framework. I’ll show how to build an agent using GPTScript to modernize testing in Kubernetes, making it faster, easier, and more adaptive to the constantly changing environment.

This blog post is built up to my talk at DevOpsDays Kerala, where I’ll speak on “Is Your Testing Stuck In the Past? Modernizing Kubernetes Testing With GPTScript and Testkube”

Introduction To Testkube

Testkube is a Kubernetes native testing framework designed to simplify testing. It Allows you to orchestrate and automate complex testing workflows using different testing tools, all from a single intuitive UI.

Unlike other testing frameworks and tools, Testkube executes your tests within the Kubernetes clusters, ensuring a secure and production-like environment and thus improving the reliability of your test outcomes.

Testkube integrates with all popular testing tools, including k6, Cypress, and Postman, to name a few. Testkube allows you to write Test Workflows that are single yaml file configurations to define your entire test requirement. Further, Testkube allows you to combine tests for any of these tools in any way required to perform your system tests.

With such versatility, Testkube makes testing in Kubernetes a breeze. By integrating it with GPTScript, the process becomes much easier and smarter.

Building a Testkube Agent Using GPTScript

GTPScript is a scripting language that automates your interactions with an LLM using natural language. It uses natural language syntax, which is easy to understand and implement. One of the highlights of GPTScript is its ability to mix these natural language prompts with traditional scripts, making it extremely flexible and versatile.

Use Case

To understand the versatility of GPTScript to build agents and use it to build one for Testkube, here’s what we’ll do. On a Kubernetes cluster, we’ll deploy a simple Nginx application. The application simply prints “Hello Kerala”. We’ll build the Testkube Agent to generate a Postman Test Workflow to test the application's response. We will then update the application to print “Hello World” - upon which the Postman test will fail. We’ll ask the Testkube agent to figure out why it failed and update the Test Workflow to fix the failing test.

Note: Ensure you’re using the latest version of GPTScript.

Pre-requisites

• An OpenAI API Key.
• GPTScript configured.
• Kubernetes cluster - we’ll be using kind.
• Testkube account and a Testkube Environment configured with Testkube Agent.
• Testkube CLI and kubectl, helm configured on your system.

Configuring Testkube

Follow the steps mentioned in the Testkube documentation to install and configure Testkube on your Kubernetes cluster. Once configured, you can access the Testkube dashboard at testkube.io to connect with your cluster and create test workflows.

Install and configure the Testkube CLI and log in to your Testkube account. To do that, execute the “testkube login” command to initiate the login process. You can choose “GitHub” to log in to the account.

After that, select the cluster your default organization followed by the environment where you’ve configured your cluster; in this case, I have a “local-kind” cluster configured.

Once successfully logged in, you can use the Testkube CLI to perform various operations. You can learn more about the Testkube CLI by running the “testkube –help” command.

Writing GPTScript

The first step is to create the GPTScript for the Testkube assistant. Since GPTScript is written primarily in natural language, writing the script to perform code review is easy. Below is the testkube.gpt file.

Name: Testkube
Description: A tool to help you perform testing of your application on your Kubernetes clusters using Testkube.
Context: learn-testkube, learn-kubectl
Tools: sys.exec, sys.http.html2text?, sys.find, sys.read, sys.write, github.com/gptscript-ai/browse-web-page
chat:true

You are an assistant for Testkube and help the user create, manage and execute test workflows. You can also perform kubernetes related tasks.

Rules
1. Access the testkube workflow docs at https://docs.testkube.io/articles/test-workflows and remember the latest specification to create testworkflows.
2. Use testkube CLI to interact with Testkube.
3. Use kubectl CLI to interact with the Kubernetes cluster.
4. Based on the user's request, perform actions on the Kubernetes cluster and create, manage, delete test workflows.

---

Name: learn-testkube
Description: A tool to help you learn testkube cli
#!/bin/bash
testkube --help
testkube create --help
testkube create testworkflow --help
testkube run --help

---

Name: learn-kubectl
Description: A tool to help you learn k8s and related commands
#!/bin/bash

CMDS="kubectl helm"
echo 'The additional CLI commands are available locally, use the `exec` tool to invoke them:'
for i in $CMDS; do
    if [ -e "$(command -v $i)" ]; then
    	echo '  ' $i
    fi
done

Let us understand what this file does:

• We create a Testkube Assistant and define the tools and contexts it has access to: learn-testkube and learn-kubectl in this case.
• We enable chat mode and describe the guidelines for the assistant to follow.
• For the learn-testkube context, we provide access to the testkube CLI.
• For the learn-kubectl context, we provide access to kubectl, helm CLI.
• Further, we also wrote down the rules for it and provided it with access to the Testkube Test Workflow documentation so that it could learn the latest specifications.

You can find this GPTScript in our examples folder.

Executing the GPTScript

To execute this script, you must first configure the “OPENAI_API_KEY” environment variable.

Execute the script using the following command:

gptscript testkube.gpt

This will initialize the assistant and start in chat mode.

Note: Since this is an AI assistant, the responses you receive will differ from what is shown here, even if you provide the same prompts. Hence, you might have to take some extra steps/prompts to get this working.

Testing The Pod’s Response

Since the agent has access to kubectl, let's first check the pods running in the default namespace.

We then check for the associated services.

With the pod and service running, let’s ask it to make a Curl request to the hello-world-deployment pod to verify the response it gets.

The pod gives the response, “Hello Kerala”. With this, we have verified that the pods and services are running fine, and we are getting the expected response. Let’s now go ahead and create a test.

Creating Testkube Test Worfklow

Testkube allows you to write single YAML file specifications to define your test workflow. Let us ask our Testkube AI Agent to write a Postman Test Workflow to test the response.

Below is the test-workflow yaml file it generates:

apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
  name: postman-hello-world-validation
spec:
  content:
	files:
  	- path: hello-world.postman_collection.json
    	content: |
      	{
        	"info": {
          	"name": "Hello World Validation",
          	"_postman_id": "12345-67890-abcdef",
          	"description": "Collection to validate the response of the hello-world service",
          	"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
        	},
        	"item": [
          	{
            	"name": "Validate Hello World Response",
            	"request": {
              	"method": "GET",
              	"header": [],
              	"url": {
                	"raw": "http://hello-world-service.default.svc.cluster.local",
                	"protocol": "http",
                	"host": [
                  	"hello-world-service",
                  	"default",
                  	"svc",
                  	"cluster",
                  	"local"
                	]
              	}
            	},
            	"response": []
          	}
        	],
        	"event": [
          	{
            	"listen": "test",
            	"script": {
              	"exec": [
                	"pm.test(\"Status code is 200\", function () {",
                	"	pm.response.to.have.status(200);",
                	"});",
                	"pm.test(\"Response is 'Hello, World!'\", function () {",
                	"	pm.response.to.have.body(\"Hello, World!\");",
                	"});"
              	],
              	"type": "text/javascript"
            	}
          	}
        	]
      	}
  container:
	resources:
  	requests:
    	cpu: 256m
    	memory: 128Mi
	workingDir: /data/repo
  steps:
	- name: Run Postman Test
  	run:
    	image: postman/newman:5-alpine
    	args:
      	- run
      	- hello-world.postman_collection.json

We can see that it has understood how Postman tests are to be written, so it created a collection along with a test to validate the response from the hello-world-service to be “Hello Kerala”. It followed the latest guidelines on Testkube Test Workflow documentation and created a specification.

We ask it to save it in a yaml file and use that file to create the Test Workflow.

Once the Test Workflow is created, we can also validate that in our Testkube dashboard.

Executing Test Workflow

We’ll now ask it to execute the test workflow, and we’ll validate it on the Testkube dashboard as well.

The test has successfully passed. The response is “Hello Kerala”, which was validated by our Postman test.

Update Application & Re-test

Now, let us upgrade the image of our Nginx application. This upgraded image will give a response of “Hello World” instead of “Hello Kerala”. Since our agent has the knowledge and access to kubectl, we’ll ask it to update the image too.

We execute the Postman Test Workflow again to the response.

As expected, the test failed because the response is “Hello World” and our Postman test is looking for “Hello Kerala”.

Fixing The Test Workflow

The Testkube AI agent is smart enough to figure out why the test failed and provide a resolution, let’s see how that works.

We can see that it very accurately figured out why the test failed. It suggests a few next steps, which include:

• Testing the deployment’s image,
• Checking the service to see if that is routing to the right deployment, and
• Updating the Postman test accordingly.

We ask it to fix the Postman test and update the Test Workflow.

It updates the Test Workflow and executes it, and this time, it should pass.

As expected, the updated Postman Test Workflow passed this time as it was now looking for a “Hello World” response, and it got “Hello World”.

And that’s how we can “talk” to an AI agent to smartly and effortlessly manage our Testkube Test workflows. It was similar to talking to an intern and asking them to create and manage tests. The difference is that I interacted with an LLM, and the process was faster and smarter.

However, creating a GPTScript in the first place requires planning and attention to detail. What assistant do you want to build? What are its capabilities? What are the tools it should support? These are some questions that you need to ask yourself before writing one. And for basic tasks, this can be too much of an effort.

That’s where Clio comes in.

Clio - The AI-Copilot You Need

Clio is an AI-powered copilot designed to assist with DevOps tasks using command-line interface (CLI) agents. It harnesses OpenAI's capabilities to deliver intelligent, context-aware support directly from your terminal, simplifying the management of complex workflows and multi-cloud environments.

Key Features

• Preconfigured Agents: Clio includes built-in agents for Azure, GCP, AWS, GitHub, and Kubernetes, making it well-suited for most common platforms and use cases.
• Customizable: You can create and integrate your agents using GPTScript to tailor Clio to your needs. We can also configure Clio to use the testkube agent we built in this post.
• Extendable: Clio’s features can be easily expanded and customized with minimal coding, making it accessible even to those with limited coding experience.

Installing Clio is as simple as running

and you can use your GitHub account and get access to OpenAI - without an OpenAI account - or bring your own OpenAI API key.

You can easily configure Clio to use the Testkube agent we built. Simply follow the add

in your testkube.gpt file and move it to folder, and run clio.

You can see that we now have our Testkube agent as part of the clio. Do read my review of Clio. Read it to understand the nuances.

Overall, GPTScript is a wonderful and easy-to-use tool for building AI Agents. We saw that in this blog post as we created a Testkube AI agent to streamline and automate our testing.

Clio is a good sidekick if you’re into DevOps and perform many tasks using various cloud providers, Kubernetes and GitHub. While I’ve shared my opinions here, go ahead and give Clio a try and see how it fares.

For the latest updates on Clio, follow @Clio_ai on X.