Exploring K6 for performance testing

Note: Running blog, will complete this blog by Dec 1st

Background

I am working in a startup who’s techstack based on javascript (nodejs + angular +PGSQL+ mongo). As system is scaling up we need build performance suites. Based on my previous experience I built performance suite using jmeter for which later I regretted for below reasons.

• Jmeter is not suited for API's which has complex and dynamic payloads. Generating dynamic and complex payloads using groovy is possible but tricky in some scenarios.

• Bad groovy editor of Jmeter makes it difficult to find errors and write complex code and it is time consuming.

• JMeter is resource-intensive. Need to go to c5a.4xlarge (16 cpu, 32 GB) to simulate 2K Users. Running from local even with 500 users is not possible (on Mac M2 with 16 Gb ram).

• Maintenance of scripts is a problem, due to language and tool barrier.

Reasons I decided to try K6

• Developer-Centric, Code-First Approach. k6 uses a code-first approach (JavaScript), making it familiar for developers and easy to version control as scripts. I don’t want to be solo person who need to maintain these performance tests. More colleberation will bring more coverage.

• k6 is designed to be efficient in resource usage, capable of generating a high load from minimal resources, even from local machines.

• Easy to configure real time traffic scenarios.

• Integration with grafana

Installation & Getting started

Mac

brew install k6

Basic script to hit url

import http from 'k6/http'
import { sleep } from 'k6'

export const options = {
    vus: 10,
    iterations: 40
}

export default function () {
    http.get('http://test.k6.io');
    sleep(1);
}

• VUS : Running 10 virtual users.

• iterations : Running the function for 40 times.

• we use sleep in the script to control no of http_reqs per second.

How to run test?

k6 run script.js

Sample Result

• iterations : Tells how many times function executed

• if number of iterations & http_reqs have different count, then some redirections are happening

• we use sleep in the script to control no of http_reqs per second.

• http_req_duration is important parameter to analyse response times

Debugging http requests

k6 run --http-debug script.js //prints response info omitting body
k6 run --http-debug="full" script.js //print entire response

Passing environment variable

k6 run -e BASE_URL=https://www.google.com
//accessing
console.log(__ENV.BASE_URL)

Load Testing types

How to simulate ramp-up/hold-time/ramp-down?

we can simulate ramp-up/hold-time/ramp-down in stages. Below options will ramp up 50 users in 5m, then 100 users in 30 min and ramp down all users to 0 in last 5 mins

import http from 'k6/http'
import { sleep } from 'k6'

export const options = {
    stages: [
        {duration : '5m', target: 50},
        {duration : '30', target: 100},
        {duration : '5m', target: 0}
    ]
}
export default function () {
    http.get('http://test.k6.io');
    sleep(1);
}

Handling Authentication for multiple scenarios

To structure k6 tests with different APIs in separate files while passing an authentication token, you can manage shared data (like tokens) centrally and reuse it across test scripts. Here's a practical example:

File Structure

Suppose you have the following APIs:

1. User API

2. Product API

3. Order API

The file structure looks like this:

k6-tests/
│
├── main.js         # Main entry point
├── auth.js         # Authentication logic
├── userApi.js      # User API test logic
├── productApi.js   # Product API test logic
└── orderApi.js     # Order API test logic

Authentication Logic (auth.js)

This file handles the generation or retrieval of the authentication token.

import http from 'k6/http';

export function getAuthToken() {
    const res = http.post('https://api.example.com/auth/login', JSON.stringify({
        username: 'testuser',
        password: 'password123',
    }), {
        headers: { 'Content-Type': 'application/json' },
    });

    if (res.status !== 200) {
        throw new Error(`Failed to authenticate: ${res.body}`);
    }

    const token = JSON.parse(res.body).token;
    return token; // Return the token
}

User API Script (userApi.js)

This script imports the getAuthToken function and uses the token in requests.

import http from 'k6/http';
import { check } from 'k6';

export function userApiTest(token) {
    const res = http.get('https://api.example.com/users', {
        headers: { Authorization: `Bearer ${token}` },
    });

    check(res, {
        'status is 200': (r) => r.status === 200,
        'response contains users': (r) => r.body.includes('users'),
    });
}

Product API Script (productApi.js)

Similar structure, reusing the token.

import http from 'k6/http';
import { check } from 'k6';

export function productApiTest(token) {
    const res = http.get('https://api.example.com/products', {
        headers: { Authorization: `Bearer ${token}` },
    });

    check(res, {
        'status is 200': (r) => r.status === 200,
        'response contains products': (r) => r.body.includes('product'),
    });
}

Order API Script (orderApi.js)

javascriptCopy codeimport http from 'k6/http';
import { check } from 'k6';

export function orderApiTest(token) {
    const res = http.post('https://api.example.com/orders', JSON.stringify({
        productId: 1,
        quantity: 2,
    }), {
        headers: {
            'Content-Type': 'application/json',
            Authorization: `Bearer ${token}`,
        },
    });

    check(res, {
        'status is 201': (r) => r.status === 201,
        'order created successfully': (r) => r.body.includes('orderId'),
    });
}

Main Entry File (main.js)

This file fetches the token once and passes it to all test scripts.

import { getAuthToken } from './auth.js';
import { userApiTest } from './userApi.js';
import { productApiTest } from './productApi.js';
import { orderApiTest } from './orderApi.js';

const authToken = getAuthToken(); // Get the token once

export const options = {
    scenarios: {
        user_api: {
            executor: 'constant-vus',
            vus: 5,
            duration: '30s',
            exec: 'userApiScenario',
        },
        product_api: {
            executor: 'constant-vus',
            vus: 5,
            duration: '30s',
            exec: 'productApiScenario',
        },
        order_api: {
            executor: 'constant-vus',
            vus: 5,
            duration: '30s',
            exec: 'orderApiScenario',
        },
    },
};

export function userApiScenario() {
    userApiTest(authToken);
}

export function productApiScenario() {
    productApiTest(authToken);
}

export function orderApiScenario() {
    orderApiTest(authToken);
}

How to set a constant number of virtual users (VUs) making requests per second ?

In k6, to set a constant number of virtual users (VUs) making requests per second, you use the constant-arrival-rate executor. This allows you to specify the exact number of iterations (requests) to execute per second, rather than managing VUs explicitly.

Example: Constant Requests Per Second

Here is an example script that sends a constant number of requests per second:

import http from 'k6/http';

export const options = {
  scenarios: {
    constant_rate: {
      executor: 'constant-arrival-rate',
      rate: 10, // 10 iterations (requests) per second
      timeUnit: '1s', // The rate is per second
      duration: '1m', // Run the test for 1 minute
      preAllocatedVUs: 20, // Preallocate up to 20 VUs for handling requests
      maxVUs: 50, // Allow up to 50 VUs to scale if needed
    },
  },
};

export default function () {
  http.get('https://test.k6.io');
}

Explanation of Options:

• executor: 'constant-arrival-rate': Ensures a fixed number of iterations (requests) per unit of time.

• rate: The number of iterations to start per second (or per time unit defined in timeUnit).

• timeUnit: Specifies the time unit for the rate (default is '1s' for seconds).

• duration: The total duration of the test.

• preAllocatedVUs: The number of VUs allocated upfront to handle the workload. This should typically be greater than or equal to rate.

• maxVUs: The maximum number of VUs that can be allocated if needed.

Key Considerations

1. Preallocate Enough VUs:

   • Ensure preAllocatedVUs is sufficient to handle the rate. If requests take longer to complete, k6 might need to scale VUs up to the maxVUs limit.

   • For instance, if each request takes 2 seconds and you're sending 10 requests per second, at least 20 VUs are required.

2. Handling Scaling:

   • If the allocated VUs are not sufficient, k6 will scale up to maxVUs. Monitor this to avoid hitting the limit.

3. Use Thresholds:

   • Add thresholds to monitor performance and ensure that the service meets your requirements.

    javascriptCopy codethresholds: {
      'http_req_duration': ['p(95)<200'], // 95% of requests should complete within 200ms
    },

4. Avoid Overloading the System:

   • Start with a smaller rate and gradually increase it to avoid overwhelming the server.

Sample Service Level Objectives(SLO)

Availability :

The application will be available 99.8% of the time.

Response Time:

• 90% of the responses are within 0.5s of recieving requests.

• 95% of the responses are within 0.9s of recieving requests.

• 99% of the responses are within 2.5s of recieving requests.