Anatomy of a test

In this article, we will break down the key components of a test in Qapir, explain what each part is responsible for, and discuss how to write tests using this powerful tool.

Test Definition

In Qapir, tests are defined using YAML files, which provide a simple, readable structure for describing the test steps. A test in Qapir consists of a sequence of actions and validations that check if the API behaves as expected.

Test Name and Test Step Description

Every test has a test-name and includes step-descriptions, which help in organizing and understanding the test.

name: Test name

test_steps:

  - description: Description of step-1
    type: http
    expected_http_status: 201
    ...

  - description: Description of step-2
    type: http
    expected_http_status: 200
    ...

Test Steps

Each test in Qapir is composed of a series of steps. These steps define the sequence of actions that the test will perform and are executed in order. Steps generally consist of two parts

• Action: Define what the test should do, such as making an HTTP request to an API endpoint.
• Validation: Verify the results of the action, ensuring the API's response meets expected criteria, such as status codes, response time, or content.

Action

An action might define a simple HTTP request to an API, including details like the URL, method (GET, POST, etc.), headers, and body data. Here’s an example of a GET request where url is https://api.example.com/v1/resource?key=value and header Authorization has value Bearer access_token

...

test_steps:
  - description: Make a GET request
    type: http
    method: GET
    url: "https://api.example.com/v1/resource"
    headers:
        Authorization: "Bearer access_token"
    query_params:
        key: value
...

Validation

A validation checks whether the result of an action is correct. In Qapir, you can validate a response by checking attributes such as the status-code, response body, and headers. The most basic validation is to check that received http-status marches expected. Continuing with our previous example, adding expected_http_status: 200 will make validate http-status.

...

test_steps:
  - description: Make a GET request
    type: http
    method: GET
    url: "https://api.example.com/v1/resource"
    headers:
        Authorization: "Bearer access_token"
    query_params:
        key: value
    expected_http_status: 200
...

Response body validation is handled using Qapir's robust test syntax called qtl. For more details, please visit this page.

Environment Variables

Environment variables are environment-specific attributes, such as a Base URLs, IDs, etc. Environment variables are defined in environment-files and are available to all tests. Tests can also declare new environment variables and update existing ones at runtime. Note that Qapir does not persist environment variables that were declared or updated at runtime; changes are only effective during the test/suite run. Here's an example of an environment-file:

# file-location: .qapir/environments/local.yml

environment_variables:
    url: "http://127.0.0.1:7070"
    string_variable: "string_value"
    number_variable: 200
    bool_variable: true

Here's how environment-variables are accessed in tests:

name: Send a GET request to '${env.url}?force=${env.bool_variable}' and expect http-status ${env.number_variable}

test_steps:
  - description: Make a GET request
    type: http
    method: GET
    url: ${env.url}
    headers:
        Custom-Header: ${env.string_variable}
    query_params:
        force: ${env.bool_variable}
    expected_http_status: ${env.number_variable}
...

Test Variables

Test variables function similarly to environment variables, with the key difference being that they are defined and visible only within a specific test. The declaration, assignment, and referencing of test variables are managed by qtl. For more information, please visit this page.

Test Paramemters

To parametrize tests, include a test_parameters object in your test scenario. In this example, we'll add a parameter named product, and then reference it within the test as ${param.product}:

name: Test with paramemter ${param.product}

test_parameters:
  product:
    - product_1
    - product_2
    - ${env.default_product}

test_steps:
    - description: Step description
      type: http
      url: ${env.url}/objects/${param.product}
...

Note that test-parameters is a map, where key is a string and value is an array. It is required that all keys point to arrays of the similar lengths.

Test Parallelism

When you run a test-suite, Qapir divides all tests into two groups: parallel and sequential. Sequential tests are executed in strict order, based on how they appear in the suite-file. Parallel tests all run in their own threads. Here's how you mark a test as parallel:

name: Parallel test

parallel: true

test_steps:
    - description: Step description
      type: http
...

By default all tests are sequential. Note, all instances of a parallel test with parameters are executed sequentially.

Retries

Qapir can retry a test-step and the whole test-scenario until they succeed. Success means that every data-extraction and data-validation has not failed. In thie following example the total count of attempts is 3 (1 main attempt + 2 retries).

name: Send a GET request

retries: 2

test_steps:
  - description: Make a GET request
    type: http
    method: GET
    url: ${env.url}
    headers:
        Authorization: "Bearer ${env.string_variable}"
    query_params:
        force: ${env.bool_variable}
    expected_http_status: ${env.number_variable}
...

Test-step retries - total count of attempts is 3 (1 main attempt + 2 retries) with an interval of 5 seconds.

name: Send a GET request

test_steps:
  - description: Make a GET request
    retries: 2
    interval: 5s
    type: http
    method: GET
    url: ${env.url}
    headers:
        Authorization: "Bearer ${env.string_variable}"
    query_params:
        force: ${env.bool_variable}
    expected_http_status: ${env.number_variable}
...