Plugin features

This document describes the most common and useful features in the plugin that one can use for writing test scenarios easily.

🤝 Shared state

The context dictionary is used to share state within and between different scenarios. Any step can make use of the context dictionary to share information of the execution of a test with the rest of the steps. There are certain keys that are reserved for the internal use of the plugin. The following table describes those keys:

KEY NAME	DESCRIPTION
current_polling_timeout_key	The timeout for polling steps
current_polling_freq_key	The polling frequency for polling steps
init_time	The start time for validation of polling steps

However, in order to not create many cross dependencies between the steps, we do not recommend overusing the test context. Storing some constant value is preferable over complex entities.

🔁 String replacement in step definitions

PyTest BDD offers a way to set variables for test scenario steps inside of .feature files. But what if you want to use variables that are only known at test runtime? For instance, the start time of the test run?

PyTest BDD EWX offers a solution for this by way of templated variables inside of step definitions. When writing new step definitions using pytest-bdd's @given, @when and @then decorators you can mark parts of the step string as needing to be replaced at runtime with variables inside of the context dictionary (see 'Shared state' above). A hook, defined by this plugin, will then make sure to replace the annotated part of the step with a variable value from the context, using the annotation as a key.

NOTE: Since the hook runs before your test logic you can only use values that are known before your actual test runs. If you're in need of values that are not defined for you yet, you'll need to make sure the context contains them before your actual tests runs. This can be done by using other fixtures to populate the context.

The marking is done by using the $ sign to prefix the part of the string that contains the context key. It is strongly advised to use this in conjunction with an opening { and closing } to mark the beginning and ending of the key. This will increase the chance of a successful parse the most.

Example:

Then there is a datasource with id "datasource_${init_time}"

NOTE: Since the characters $, { and } are used for this string replacement functionality they are reserved characters and shouldn't be used for anything else inside step definitions. If you try to use them as regular characters you'll like experience unexpected behaviour.

🛠️ String replacement in config files

Besides the step definition strings, test files residing in the /data folder (see user manual) can also contain string replacement annotations. This can help in adding custom checks like a specific data source id that is being generated upon ingestion of these files.

Example: An xml file has a field that contains the value of the data source id that is being created upon ingestion of that file. That field can be replaced by ${datasource_id} and this can be used in the feature files to identify the data source to be validated. This can be used from custom step files by using the FILE_REPLACEMENTS_KEY field in the context dictionary.

For example:

   base_id = f"{int(TEST_TIMESTAMP):015d}"
   context[FILE_REPLACEMENTS_KEY]['datasource_id'] = base_id

Where datasource_id is the datasource id that can be used in the data configuration files (by using ${datasource_id}). Here is the step that uses this functionality:

Then I upload the file "{file_path}" where the value for "{key}" is "{value}"

where for instance, the key is datasource_id and the value is desired value which will be unique every time this step is executed.

📅 Dates relative to the test run

Fixtures with fixed dates age: a billing period that was "last month" when the scenario was written drifts ever further into the past. The ${date:<offset>|<format>} placeholder renders a timestamp relative to the day the tests started, so a data file, a configuration or a step argument can describe "the 744 hours up to today 00:00" instead of a fixed period. It is resolved before the plain ${key} replacement, everywhere that replacement applies (data table cells included; escape the | as \| there).

The anchor is 00:00 of the day the test session started, in TEST_TIMEZONE (Europe/Amsterdam, override with the AUTOMATED_TESTS_TIMEZONE environment variable). The offset is a sequence of signed terms, applied left to right:

Term	Meaning
d	Calendar days, keeping the local wall clock: -31d is midnight 31 days ago, also across a DST switch
h	Exact hours: -744h is exactly 744 hours earlier (23:00 or 01:00 local when a DST switch lies inside)
m	Exact minutes: -15m is the last quarter-hour of yesterday

The format selects the rendering:

Format	Example	Typical use
iso (default)	2024-08-01T00:00:00+02:00	Timestamps in CSV data, platform replies
utc	2024-07-31T22:00:00.000Z	Request periods
ms	1722463200000	Datapoint timestamps in timeseries steps
date	2024-08-01	Dates
local	2024-08-01 00:00:00	Local timestamps without offset

Example:

Then I expect in datasource with id "BILLING_${test_timestamp}" in channel "ACTIVE_DELIVERY_TOTAL_BILLING" data point "${date:0d|ms}", "63999.124"

Use exact terms (h, m) for the data and the period boundaries when the expected values are sums over the period: that keeps the number of intervals in the period constant across DST switches. Use calendar days (d) for things that must land on a local midnight regardless of the hour count.

A malformed offset or an unknown format raises a ValueError when the step runs, rather than reaching the platform as literal text.

✏️ Id replacement for config files

Many entities in the API depend on other entities, for instance Market Adapters need to specify a Transformation Configuration. These are usually specified in the config schemas via an id to the entity they refer to. Because these ids are only created upon creation time of the referenced entity itself they can't be provided before hand. In order to facilitate linking between these entities id templating is provided, this feature replaces templated names of entities by their corresponding ids.

Templated fields are identified by the use of the $ character and the { and } characters to surround the templated fields. The templated fields must be 2 elements separated by : where:

• The first element is the name (in snake case format) of the entity that we want to use
• The second element is the name of the entity which needs to be replaced by its corresponding id NOTE: the creation of the entities from the scenario files must respect the dependency order in which they are refered. E.g. a Market Adapter id can refer to a Transformation configuration id this way: transformationConfigurationId:${transformation_configuration:some_name_from_a_transformation_configuration_file}

⏱️ Polling for asynchronous requests

Some common testing scenarios need to execute processes in the energyworx platform which are fire-and-forget by nature e.g. ingesting a file, executing a flow. Validating the results of such scenarios can only be done by executing calls to the API frequently until getting the expected results with a timeout. The pytest-bdd-ewx plugin contains a built-in mechanism that can be used for performing periodic calls to the API. This mechanism of polling can be easily used from any scenario by using the statement: Then I do following steps with a timeout of "{timeout}" minutes and a poll frequency of "{poll_freq}" seconds Where timeout and poll_freq can be specified for the rest of the validations in that scenario. This step is not completely necessary though if you want to use the defaults defined in DEFAULT_POLLING_TIMEOUT and DEFAULT_POLLING_FREQ (see config.py). These parameters will be set in the context dictionary to be shared with the following steps.

NOTE: These will be kept for the subsequent scenarios unless overwritten by using the aforementioned statement again on that scenario.

A helper function to execute calls periodically (check_action_periodically) is available in the utils folder. For more information on how to use it consult the documentation of the function.

The following steps use the polling variables to perform periodic calls to the API:

Step	Class
I expect in "{channel_classifier}" property tag "{tag_key}", "{tag_value}" and "{is_updated}"	Datasource
I expect in "{channel_classifier}" data point "{timestamp}", "{value}"	Timeseries
I expect for datasource with id "{datasource_id}" a channel with channel classifier "{classifier_name}" which is "{source_or_not}"	Channel Classifier
I expect new audit events	Audit Events

etc. - the steps that use check_action_periodically function under the hood.

👥 Support for multiple APIs

In order to support both API versions (v1.5 and v2), we added the environment variable AUTOMATED_TESTS_API_VERSION. Currently, it supports the following values: "1.5" or "2" and it uses "1.5" as the default value.

It is also possible to use the step Given I switch to api version "{api_version}" to switch to between the API versions during runtime. This switch will be valid only for that module (python file).

Also, to make the code separation clearer, we split the steps into steps.py and steps_implementation.py files, so that the same step can be still used when running the test against any API version, and the requested implementation (v1.5 or v2) will be recognised and used during the test execution. You can read more about it here in Adding steps section.