QTL

About

The qtl syntax is used to perform validations on responses and variables. It allows you to:

1. Find data in response-payloads and variables via jsonpath, then extract it into test-variables.
2. Perform various types of validation on extracted data.

qtl has several reserved keywords that allow you difine logic for data-extraction, variable-assignment and data-validation.

Every qtl statement ends with a ;

Data-extraction

SELECT - instructs Qapir to find data via jsonpath in a response-body, response-headers or in a variable.

AS - specifies how the data found by SELECT should be stored: as a test-variable, or as an environment-variable.

SELECT_ALL - same as SELECT, but can only be invoked on an array-data for further iterative validation of all array-elements.

FROM - defines data-source, it can be either a response-body, response-headers, or a test/environment-variable.

AND - a keyword that allows you to execute multiple data-extractions from the same source.

The following example finds data in response-body by the following jsonpath statements: $.data.number and $.data.string, then saves found data as ${var.number} and ${var.string} test-variables respectively:

SELECT 
  $.data.number AS ${var.number} AND
  $.data.string AS ${var.string}
FROM 
  ${response.body};

Variable-assignment

SET - this directive is used to set/update a test/environment-variable outside of SELECT statement.

The following example sets two test-variables ${var.string} and ${var.array} with values your_string_value and [1, 2, 3] respectively:

SET
  ${var.string} "your_string_value" AND
  ${var.array} [1, 2, 3];

Data-validation

EXPECT - performs validation of a single entity.

EXPECT_ALL - performs validation of every entity in an array on which this directive is invoked.

IF_EXISTS - a directive that instructs Qapir to perform validation of an entity only if it exists - without this directive, validation on a null-entity fails.

Validation-terms

-type - checks whether entity is one of string, number, bool, array, object

-eq - checks whether entity is equal to another entity

-neq - checks whether entity is not equal to another entity

-lt - works with numbers only, checks whether entity is less than another entity

-le - works with numbers only, checks whether entity is less than or equal another entity

-gt - works with numbers only, checks whether entity greater than another entity

-ge - works with numbers only, checks whether entity greater than or equal another entity

-len - works with strings and arrays, checks whether length of one entity is <, <=, >, >= or equal to length of another entity

-in - checks whether entity is part of an array

-nin - checks whether entity is not part of an array

-contains - works with strings and arrays, checks whether a string contains a given substring, or if an array contains a given element

-ncontains - works with strings and arrays, checks whether a string does not contain a given substring, or if an array does not contain a given element

-exists - checks whether entity exists

-nexists - checks whether entity does not exist

Array-validation

Qapir has a directive that simplifies validation of arrays - EXPECT_ALL, which works in pair with SELECT_ALL

Imagine you need to validate whether all elements of a json-array contain expected values.

Given the following json with job-objects:

{
  "data": [
    {
      "name": "job_1",
      "status": "done"
    },
    {
      "name": "job_2",
      "status": "in_progress"
    }
  ]
}

Check that every job-object has fields name and status, that both fields are of type string, and status is one of [in_progress, done]

Here's how to write such statement:

SELECT
  $.data AS ${var.data}
FROM
  ${response.body};
  
SELECT_ALL
  $.name AS ${var.iterable_job_name} AND
  $.status AS ${var.iterable_job_status}
FROM
  ${var.data}
EXPECT_ALL
  ${var.iterable_job_name} -type string AND
  ${var.iterable_job_status} -type string AND
  ${var.iterable_job_status} -in ["in_progress", "done"];

First, we need to get the array with SELECT and save it as ${var.data}. This step is needed because SELECT_ALL can only be called on an array.

Next, when directive SELECT_ALL is executed, Qapir under the hood creates 2 test-variables with name iterable_job_name and 2 test-variables with name iterable_job_status, since there are 2 job-objects.

Finally, EXPECT_ALL runs validations on every instance of iterable_job_name and iterable_job_status.

Alternatively, instead of EXPECT_ALL we could use EXPECT term and reference every instance of iterable_job_name and iterable_job_status by appending #{index} to their variable-names.

Although this statement is longer and riskier to use (we might not always know array's length), it is also valid:

SELECT
  $.data AS ${var.data}
FROM
  ${response.body};
  
SELECT_ALL
  $.name AS ${var.iterable_job_name} AND
  $.status AS ${var.iterable_job_status}
FROM
  ${var.data}
EXPECT
  ${var.iterable_job_name#0} -type string AND
  ${var.iterable_job_name#1} -type string AND
  ${var.iterable_job_status#0} -type string AND
  ${var.iterable_job_status#1} -type string AND
  ${var.iterable_job_status#0} -in ["in_progress", "done"] AND
  ${var.iterable_job_status#1} -in ["in_progress", "done"];

Optional fields

There are cases where a field is expected to be missing in one case, and present in another. Continuing with the previous example, let's imagine that our jobs-response looks like this:

{
  "data": [
    {
      "name": "job_1",
      "status": "done",
      "comment": "some random comment"
    },
    {
      "name": "job_2",
      "status": "in_progress"
    }
  ]
}

Notice that one job has field comment, while the other one does not. This might be the case with the API you are testing, and Qapir has a directive IF_EXISTS that instructs the test to only validate comment if it exists.

Here's a complete statement that would achieve this:

SELECT
  $.data AS ${var.data}
FROM
  ${response.body};
  
SELECT_ALL
  $.name AS ${var.iterable_job_name} AND
  $.status AS ${var.iterable_job_status} AND
  $.comment AS ${var.iterable_job_comment}
FROM
  ${var.data}
EXPECT_ALL
  ${var.iterable_job_name} -type string AND
  ${var.iterable_job_status} -type string AND
  ${var.iterable_job_status} -in ["in_progress", "done"] AND
  IF_EXISTS ${var.iterable_job_comment} -type string;