Plugin features

This document describes the most common and useful features in the plugin that one can use for writing test scenarios easily.

🤝 Shared state

The context dictionary is used to share state within and between different scenarios. Any step can make use of the context dictionary to share information of the execution of a test with the rest of the steps. There are certain keys that are reserved for the internal use of the plugin. The following table describes those keys:

KEY NAME	DESCRIPTION
current_polling_timeout_key	The timeout for polling steps
current_polling_freq_key	The polling frequency for polling steps
init_time	The start time for validation of polling steps

However, in order to not create many cross dependencies between the steps, we do not recommend overusing the test context. Storing some constant value is preferable over complex entities.

🔁 String replacement in step definitions

PyTest BDD offers a way to set variables for test scenario steps inside of .feature files. But what if you want to use variables that are only known at test runtime? For instance, the start time of the test run?

PyTest BDD EWX offers a solution for this by way of templated variables inside of step definitions. When writing new step definitions using pytest-bdd's @given, @when and @then decorators you can mark parts of the step string as needing to be replaced at runtime with variables inside of the context dictionary (see 'Shared state' above). A hook, defined by this plugin, will then make sure to replace the annotated part of the step with a variable value from the context, using the annotation as a key.

NOTE: Since the hook runs before your test logic you can only use values that are known before your actual test runs. If you're in need of values that are not defined for you yet, you'll need to make sure the context contains them before your actual tests runs. This can be done by using other fixtures to populate the context.

The marking is done by using the $ sign to prefix the part of the string that contains the context key. It is strongly advised to use this in conjunction with an opening { and closing } to mark the beginning and ending of the key. This will increase the chance of a successful parse the most.

Example:

Then there is a datasource with id "datasource_${init_time}"

NOTE: Since the characters $, { and } are used for this string replacement functionality they are reserved characters and shouldn't be used for anything else inside step definitions. If you try to use them as regular characters you'll like experience unexpected behaviour.

🛠️ String replacement in config files

Besides the step definition strings, test files residing in the /data folder (see user manual) can also contain string replacement annotations. This can help in adding custom checks like a specific data source id that is being generated upon ingestion of these files.

Example: An xml file has a field that contains the value of the data source id that is being created upon ingestion of that file. That field can be replaced by ${datasource_id} and this can be used in the feature files to identify the data source to be validated. This can be used from custom step files by using the FILE_REPLACEMENTS_KEY field in the context dictionary.

For example:

   base_id = f"{int(TEST_TIMESTAMP):015d}"
   context[FILE_REPLACEMENTS_KEY]['datasource_id'] = base_id

Where datasource_id is the datasource id that can be used in the data configuration files (by using ${datasource_id}). Here is the step that uses this functionality:

Then I upload the file "{file_path}" where the value for "{key}" is "{value}"

where for instance, the key is datasource_id and the value is desired value which will be unique every time this step is executed.

✏️ Id replacement for config files

Many entities in the API depend on other entities, for instance Market Adapters need to specify a Transformation Configuration. These are usually specified in the config schemas via an id to the entity they refer to. Because these ids are only created upon creation time of the referenced entity itself they can't be provided before hand. In order to facilitate linking between these entities id templating is provided, this feature replaces templated names of entities by their corresponding ids.

Templated fields are identified by the use of the $ character and the { and } characters to surround the templated fields. The templated fields must be 2 elements separated by : where:

• The first element is the name (in snake case format) of the entity that we want to use
• The second element is the name of the entity which needs to be replaced by its corresponding id NOTE: the creation of the entities from the scenario files must respect the dependency order in which they are refered. E.g. a Market Adapter id can refer to a Transformation configuration id this way: transformationConfigurationId:${transformation_configuration:some_name_from_a_transformation_configuration_file}

⏱️ Polling for asynchronous requests

Some common testing scenarios need to execute processes in the energyworx platform which are fire-and-forget by nature e.g. ingesting a file, executing a flow. Validating the results of such scenarios can only be done by executing calls to the API frequently until getting the expected results with a timeout. The pytest-bdd-ewx plugin contains a built-in mechanism that can be used for performing periodic calls to the API. This mechanism of polling can be easily used from any scenario by using the statement: Then I do following steps with a timeout of "{timeout}" minutes and a poll frequency of "{poll_freq}" seconds Where timeout and poll_freq can be specified for the rest of the validations in that scenario. This step is not completely necessary though if you want to use the defaults defined in DEFAULT_POLLING_TIMEOUT and DEFAULT_POLLING_FREQ (see config.py). These parameters will be set in the context dictionary to be shared with the following steps.

NOTE: These will be kept for the subsequent scenarios unless overwritten by using the aforementioned statement again on that scenario.

A helper function to execute calls periodically (check_action_periodically) is available in the utils folder. For more information on how to use it consult the documentation of the function.

The following steps use the polling variables to perform periodic calls to the API:

Step	Class
I expect in "{channel_classifier}" property tag "{tag_key}", "{tag_value}" and "{is_updated}"	Datasource
I expect in "{channel_classifier}" data point "{timestamp}", "{value}"	Timeseries
I expect for datasource with id "{datasource_id}" a channel with channel classifier "{classifier_name}" which is "{source_or_not}"	Channel Classifier
I expect new audit events	Audit Events

etc. - the steps that use check_action_periodically function under the hood.

👥 Support for multiple APIs

In order to support both API versions (v1.5 and v2), we added the environment variable AUTOMATED_TESTS_API_VERSION. Currently, it supports the following values: "1.5" or "2" and it uses "1.5" as the default value.

It is also possible to use the step Given I switch to api version "{api_version}" to switch to between the API versions during runtime. This switch will be valid only for that module (python file).

Also, to make the code separation clearer, we split the steps into steps.py and steps_implementation.py files, so that the same step can be still used when running the test against any API version, and the requested implementation (v1.5 or v2) will be recognised and used during the test execution. You can read more about it here in Adding steps section.