Rules

Rules

A rule is a piece of Python code that is applied on a channel of a datasource which can validates, estimate, calculate, forecast or do anything that is needed to the ingested data. Rules provide the flexibility to change the behavior of your processes without depending on platform release changes.

Rules mainly process timeseries data of a given datasource, but they also have access to tags, channels, channel classifiers and time-slice groups. Rules can operate on timeseries coming from an ingestion but they can also work on already stored data.

A flow consists of sequences of rules. In a flow, a pandas dataframe is carried over from the first rule to the last rule. The dataframe starts with the timeseries data of the source channel together.

All flows are made up of one or more rules, which can be found under the ‘Rules & Algorithms’ section in the Energyworx platform, separated by rule type:

Within the platform, each rule has a specific rule type. The rule types can be customized and serve the purpose of categorizing rules that behave in a similar manner or are used for a similar process. New rule types can be introduced.

This page enables you to search for a specific rule, view its source code, and make adjustments if necessary.

This image shows a preview of a rule and how a user would go about accessing it:

The following sections describe the 2 main types of rules:

• Transform rules: they are used in Transformation Configurations. (See Transformation Configurations for more information)
• Flow rules: they are used in Flow Designs. (See Flow Designs for more information)

Transform rules

Transform rules are rules that ‘transform’ incoming data. They map the incoming data into any attribute dynamically configurable in the Transformation configurations (e.g. tag names, timestamps of values of channels, etc.)

Examples of transformations can range from simply dividing all incoming numerical data by a factor 1000 or adding a prefix to incoming strings; to mapping incoming values to entirely different values or retrieving specific portions of incoming strings with Regex. Transform rules can be chained together to create transformations that consist of multiple steps. So, for example, if one has the string 'WEATHER_STATION_XXX_KNMI' with 'XXX' being an ID, and wishes for this to be transformed to 'KNMI_XXX', then this result can be achieved using a combination of a rule to retrieve the ID and another rule to add the 'KNMI_' prefix.

As transform rules are applied on incoming data, they are the only type of rule that can be used within Transformation Configurations. While transform rules can be used within flow designs, it is very uncommon for them to be, due to transform rules being required to inherit a different base class than rules of other rule types (TransformRule instead of FlowRule). This separate base class lacks many of the methods that FlowRule has, making it hard to use within flows.

An example of a simple transform rule can be found below:

from ewx_public.transform_rule import TransformRule


class AddPrefixSuffix(TransformRule):
    def apply(self, string_value, prefix_suffix_flag, **kwargs):
        """
        Rule that adds a specified `string_value` as either a prefix or suffix to the channel data.

        Args:
            string_value (str): String to add.
            prefix_suffix_flag (str): Add `string_value` as `prefix` or `suffix`.

        Returns:
            pandas.Series

        """

        # Add prefix or suffix
        index = self.dataframe.dropna().index
        if prefix_suffix_flag == 'prefix':
            result = self.dataframe[self.dataframe.columns[0]][index].apply(lambda x: string_value + str(x))
        else:
            result = self.dataframe[self.dataframe.columns[0]][index].apply(lambda x: str(x) + string_value)

        # Return it
        self.dataframe[self.dataframe.columns[0]][index] = result
        return self.dataframe

Flow rules

Flow rules are rules that can be used in a flow design. Their purpose is to execute a particular action within a flow.

Working with rules

Rules go through three stages, each described on its own page:

1. Configure — create the rule in the console and define its parameters: Rule configurations
2. Implement — write the Python code blob: Rule Implementation
3. Develop and test locally — iterate with IDE support and unit tests: Local Rule Development

Creating and editing rules requires the Rule Owner permission — see IAM and Permissions.