Skip to main content

Release Notes: Version 26.09

Last updated: 6 October 2026

Pandas 2 and NumPy 2 upgrade — action required

With 26.09 the data processing environment for custom rules and market adapters moves to Pandas 2 and NumPy 2. Your rules and market adapters will only run if their code is Pandas 2 compatible — anything relying on an API removed in these versions has to be migrated first.

The most common ones: DataFrame.append() / Series.append() (use pd.concat), iteritems() (use items()), groupby(squeeze=), the squeeze, prefix, mangle_dupe_cols, error_bad_lines and warn_bad_lines arguments of read_csv() / read_excel(), and np.NaN (use np.nan). Frequency aliases such as "H" and "M" still work but now emit deprecation warnings — prefer "h" and "ME".

See the Pandas 2.x Migration Guide for before/after patterns and search expressions to scan your own rules.

What's New ✨​

User Management in the new console​

We have revamped the onboarding experience and improved user access management in the new console. Users now receive an email when they are first given access to the platform. On top of that, administrators can immediately assign users to a group upon granting them access. The view covers the namespaces where you hold the iam permission, and replaces the whitelist and IAM screens in the original console.

Other improvements to new console​

A few more pages have migrated to the new console this release:

  • Native dashboards: Dashboards in the new console use the same EQL-driven widgets as the classic console — charts, tables, lists, statistics, and audit/flow views — and read from your existing dashboard configuration, so no setup or migration is needed. In edit mode you can add, resize, and rearrange widgets on a page-filling grid, and the widget editor shows a live preview next to a syntax-highlighted query editor with one-click formatting and validation.
  • Flows, rules and decision trees: The new console now has its own flow design, rules and decision tree pages. A change to a rule’s display name propagates to every flow design that uses it automatically, previously the rule had to be removed and re-added.
  • Edit datapoint attributes directly on the chart: You can select a datapoint on the load curve in the new console and edit its attributes — reading quality included — in place, instead of locating the row in the underlying table and filtering it down. This restores the inline correction workflow that existed before reading quality became a datapoint attribute.

Richer emails from rules​

We have expanded the functionality of the feature used to send emails from rules. It now supports:

  • File attachments: send_email in the rule framework can attach a file to the message, so recipients get the data without signing in to the platform.
  • CC recipients: An email sent from a rule can CC additional stakeholders alongside its primary addressees.
  • Severity levels: A rule email can carry a severity of Critical, Warning or Info, so recipients see the urgency immediately.
  • Deep links to platform entities: Helper functions build direct links to a datasource, process or flow, so a recipient can go straight to the source of an alert.

Analytics dashboards on a datasource​

Analytics dashboards can now be attached to a datasource classifier, so they open on every datasource of that classifier alongside its charts and data — no dashboard per datasource. Filter the dashboard's dataset on the datasource_id URL parameter and its charts scope themselves to whichever datasource you opened, so one dashboard serves them all. Dashboards are attached in the classifier's Superset Dashboards section by their Embed UUID, and ewx-cli config upload carries them between namespaces. This is only available if Analytics is enabled for your environment. Analytics dashboards remain available on request at additional cost, as introduced in 26.08 — contact our service desk if you would like them switched on.

⚠️ Deprecation Notice​

  • get_tag is deprecated and will be removed in 27.03: self.datasource.get_tag() in the rule framework is deprecated, with a firm removal date: it will be removed in 27.03, six months from now. Migrate any rule that still calls it before then — it will not be maintained further in the meantime. The method returns a three-value tuple whichever combination of latest_version, active_versions and all_versions you ask for, so every caller unpacks values it never requested. Three focused methods replace it, each returning only what you asked for:
    • self.datasource.get_latest_tag_version(tag_name) — the most recent version of a tag, or None if the datasource has no such tag.
    • self.datasource.get_active_tag_versions(tag_name) — the versions currently in effect, oldest first.
    • self.datasource.get_all_tag_versions(tag_name) — every version of the tag, including superseded and removed ones.

🖥️ UI Improvements​

  • Namespace selection is reliable: The selected namespace stays in the URL as you search and navigate, survives signing out and back in, and each browser tab keeps its own — several open tabs can no longer switch it silently. (New console)
  • Settings corrections: The rule cache duration picker under Settings → Other is now labelled in seconds rather than minutes, which overstated the lifetime sixtyfold; configured values are unchanged. The "Channel heartbeat collection" switch, which never had any effect, has been removed, and the Tasks section now offers counts up to 30.
  • Tag handling fixes: A tag's "valid from" time is shown and saved in your local timezone instead of UTC, creating a tag with a name that already exists shows an error instead of silently overwriting it, clicking a timestamp attribute tag in datasource search returns the matching datasources, and editing tags on linked datasources from the virtual datasource page works again. (New console)
  • Datasource page and charts: Viewing a flow run's data shows only the channels and versions that run produced. The chart legend's "Show all" and "Hide all" toggle every channel, editing a datapoint from the Chart tab works for channels with a display name, and the start-flow picker stays anchored to its field — its source-channel field is gone, since flows run per datasource. Several visual issues on the page were corrected. (New console)
  • Dashboard configuration and widgets: Native dashboard widgets whose EQL uses double-quoted string literals no longer show "No results found.". On a datasource classifier, the dashboard identifier field is now labelled "Embed UUID" with a hint on where to find it, and the remove-dashboard button alignment is fixed.
  • Smart Integration permission gating: Create, copy, edit, delete, execute and save controls on the Market Adapters, Transformation Configurations, Trigger Schedules and Datasource Classifiers pages are hidden when you lack the matching permission, and detail and edit forms open read-only when you have read access only. (New console)
  • File management and ingestion screens: All market adapters now appear in the adapter list and the file ingestion screens rather than only the first 20, which unblocks files whose adapter was missing. A row's tags collapse into a "+N" indicator instead of being cut off mid-tag, and XML field detection no longer shows duplicate column names. (New console)
  • Sign-in environment check: The sign-in page now warns clearly when an environment's backend is unreachable, for example after deprovisioning, instead of failing with a raw error after you press a sign-in button. (New console)
  • Audit event search and API reference: Audit event search is now case-insensitive across event messages, subjects and property values, and the MDM API reference on the API Documentation page loads correctly. (New console)

⚙️ Custom Business Logic​

  • Create Excel files from a rule: A rule can now build an Excel workbook and publish it to file management or send it as an email attachment.
  • Task type and status names resolve consistently: A rule or flow design that names a task type or status now finds it regardless of capitalisation or surrounding whitespace, so create_task(..., status="open") resolves the configured "Open" status instead of failing. Where a name previously matched more than one, the platform reports the ambiguity rather than binding the task to an arbitrary one.
  • Rule authoring fixes: A custom rule referencing an undefined name — a mistyped variable or constant — now fails with a clear audit event instead of an unhandled error. A rule that imports or inherits from another rule also picks up changes to it on the next flow run, instead of running the previous version until the processing module cache expired.
  • Channel names are stored for analytics: The per-datasource channel name is now written back to the datasources table in BigQuery instead of being left empty, using the name from the transformation configuration and falling back to the classifier's display name and then its technical name — so analytics shows the same label as the console.
  • Storing to another datasource from a rule: A rule that stores timeseries or annotations on a datasource other than the one the flow runs on now records a single datasource version, instead of one extra version per store call. The datasources table and advanced search no longer show duplicate entries that differ only in their insert timestamp.
  • Collective trigger decision tree fix: A collective trigger now runs the namespace's default decision tree when the flow asks for it. The trigger was previously always published as a custom trigger, which made the crunch pipeline skip the decision tree regardless of the bypass_decision_tree argument passed to send_collective_trigger.

🔐 API & Permissions​

  • Task types and statuses: A namespace can no longer hold two task types or statuses with the same name — names are matched ignoring case and surrounding whitespace, so creating "open" where "Open" exists returns the existing one. Both can be fetched by name and deleted, unless a task, taskboard lane or flow design still references them. Task records carry their creation and update times again, and the assignee picker loads its list of users.
  • Sign-in is tied to the identity provider an account was created with: An account can only be signed in to through the provider it was created with, so revoking someone in a corporate directory such as Microsoft Entra ID cannot be worked around by using a different provider for the same address. SAML sign-in from the classic console also returns you there, rather than to the new UI.
  • API endpoint changes: The rules API gained GET /rule/list, returning rules across all types in one call; channel families can now be updated through the API; and the deprecated datasource tag-search endpoint has been removed, with no client action required.
  • Tags keep their type and reach analytics: A tag created through the API v2 datasource tags endpoint is now written to the analytics store as well, so it is returned by queries against tag data — tags from flows or API v1 already were. A tag property detected as datasource, int, float, bool or ts keeps that type instead of being downgraded to string.
  • Clearer errors and consistent values: Timeseries requests naming nonexistent channel classifiers return a descriptive 404 listing every missing channel, process monitoring payloads carrying unknown metrics or enum values are rejected with a 400, and datapoint attribute validation on channel classifiers explains itself clearly. Edit timestamps come back with microsecond precision whichever API version made the edit, and a ResourceInput parameter's resource type is no longer dropped.

🛠️ ewx-cli​

  • Bulk trigger schedule control: The new ewx-cli trigger-schedules disable command switches off every enabled trigger schedule across a project or a chosen set of namespaces and records which ones it touched; reenable switches exactly those back on. Both require their targets to be named explicitly and show the full plan before changing anything.
  • Task types and statuses are addressed by name: ewx-cli now identifies a task type or status by name rather than numeric ID, matching how taskboards and flow designs already reference them. Selecting one by ID still works, so a config repo that names a status and also selects it no longer uploads it twice. Both can now be deleted through the CLI.
  • Configuration dependency diagrams: The new ewx-cli config describe command turns a downloaded configuration set into a MermaidJS diagram showing every configuration and what it depends on. It reads the local directory only — no API calls, no permissions — takes the same filters as config download, and can write straight to a file with --output.
  • Datapoint attributes are applied on upload: ewx-cli config upload now applies the datapointAttributes of a channel classifier that already exists on the destination namespace, instead of reporting success while discarding them. Attributes can only be added, so an upload that would remove, rename, reorder or retype one — or drop an enum symbol — now fails, naming the classifier and the conflict.
  • No duplicate task statuses on upload: ewx-cli config upload no longer creates duplicate task statuses and types when a config repo ships them alongside a taskboard or flow that references them by name. The duplicates previously made every rule-created task in that namespace fail to be written.
  • Superset dashboards are applied on upload: ewx-cli config upload now applies the Superset dashboards attached to a datasource classifier instead of silently discarding them, and config download includes them so they round-trip. The block declares the classifier's complete dashboard set, so an upload replaces whatever is stored — re-uploading an older config can overwrite dashboards added since through the UI.

📚 Documentation Updates​

  • Chart presets: A new "Chart Presets" guide explains how to save the visible channels and their colours in the datasource chart as a named preset, apply or update it, and how presets differ from default visible channels.
  • pytest-bdd-ewx step reference: The "Available Gherkin Steps" page is now generated from the plugin source on every change and lists all built-in steps grouped by platform domain and entity, with a filter box to find a step by any word in its text or description.
  • Transform rule reference: The Transform Rule Implementation page now documents everything a transform rule can use — all available attributes ('self.dataframe', 'self.adapted_data', 'self.result', 'self.datasources', 'self.mapping', 'self.version', 'self.rule_logger'), how configured rule parameters reach 'apply', what to return, and how to abort an ingestion, including why a mapping whose field is missing from the file never runs its rules.
  • API documentation: The "How to use the Energyworx API" guide now describes the client credentials workflow (create a pair under Settings → Automated access, bind it to a service account via the service desk) instead of the retired JWT provider setup.

🚀 Notable Backend Changes​

  • Access rules can match an identity provider claim: An access rule can now grant groups on a namespace to everyone whose identity provider asserts a given claim — a group or role claim on a SAML assertion or an OIDC ID token — rather than only to everyone at an email domain. Rules of this kind could already be created but were stored and skipped; they are now evaluated at every sign-in. Name the claim as your provider sends it, match the value case-insensitively, and note that a multi-valued claim matches if any one value does; where a claim rule and a domain rule name the same namespace, the user gets both sets of groups. This replaces the iam_group_mapping namespace property. One limit is worth knowing: a sign-in carrying no assertion — an API token minted from client credentials — matches no claim rule and removes claim-granted access until the user next signs in through their provider, so access that must not come and go should be granted to the user directly. Every grant the rules create, change or revoke is recorded in the audit trail.
  • Ingestion routing and tagging corrected: Files ingested through a Pub/Sub payload are now tagged with their market adapter id instead of the namespace id, matching the other ingestion routes, and a file whose payload type is not mapped to an adapter falls back to the adapter id the publisher supplied rather than being stored read-only and then failing. The "Multi queue processing" setting also enables and disables labelled routing as intended — previously it was evaluated incorrectly, so labelled routing stayed off for any namespace that had saved its settings.
  • Clearer ingestion failures: An ingestion failure caused by a custom market adapter now produces an audit event naming the adapter, its technical name and version, and the file being ingested, and states whether the adapter's code could not be loaded or failed while running; the custom market adapter documentation explains the naming rules an adapter must follow. Error messages also improved for files whose column mapped to the datasource classifier is missing or empty.
  • Scheduled BigQuery exports: Empty header-only files produced by BigQuery Ingest triggers (for example when a query matches few or no rows) are now detected and skipped during ingestion, so they no longer appear in the file manager or trigger downstream processing.
  • Excel and file extension handling: Fixed a bug that prevented the generic excel market adapter from ingesting workbooks — both versions failed before any data was read. Legacy .xls and OpenDocument .ods files remain unsupported and are now rejected on upload with a message naming the accepted extensions, instead of failing part-way through ingestion. The generic CSV, XML, JSON, SOAP and Excel adapters also match a file's extension case-insensitively, so READINGS.CSV is no longer rejected.
  • Flows run where they previously failed: A flow linked to a virtual datasource is now stored and started once the aggregation finishes — previously it was never triggered, so it never ran. Datasources with tag properties of type link execute flows successfully. Flow Designs and individual rule or transformation designs also open again for older configurations whose parameters were stored without an explicit optional value, where the read path returned a null that broke the page.
  • No duplicate records or tasks: Data pipelines reliably prevent duplicate records and audit events during system upgrades or temporary interruptions. Tasks raised by several rules in the same flow run now consistently collect as items on a single task, as the rules' aggregation configuration declares, rather than being split across a varying number of tasks from run to run.
  • Richer audit events: Audit events for datasource datapoint and annotation edits now include the number of edited channels and the affected date range, and file ingestion audit events show the file identifiers when a file is uploaded and ingested simultaneously.
  • Web Application Firewall (WAF): All environments now run behind a Cloud Armor WAF policy. It blocks known CVE exploit patterns, scanner detection, and local/remote file inclusion attacks, and rate-limits repeated login attempts to 20 requests per minute per IP with a 30-minute ban on breach. No action is required on your side. Signing in with Google on a WAF-enabled environment also works again — the remote-file-inclusion rule was tripping on the Google URLs the OAuth callback carries in its scope parameter, returning a bare 403 Forbidden; both callback paths are now allow-listed.

📦 Standard Packages​

  • Billing correctness: Fixed a batch of determinant defects: Sunday and holiday timeslice tariffs now match correctly, "split by month" requests that stay within one calendar month and register reads with no reading near the period end no longer crash, and interval value and quality status now cover the same window at every channel frequency.
  • VEE validation and estimation: The compare-to-threshold, max-capacity, register-decrease and sum-check rules now open a task when they flag data — on by default, switchable per rule with create_task, and landing on the correct taskboard in every namespace instead of one fixed board. Estimated datapoints record which rule produced them in a new read-only Estimation_Method attribute alongside Quality_Status. The rules and their parameters also describe themselves consistently in the console, the Gap Check rule dropped its Data Frequency parameter (it never had any effect), and sum-check Period annotations are written without a UTC offset again.
  • Standard package dashboards: The Billing package now ships a dashboard of successful and failing calculations per day, and the Device Control package a meter events dashboard of commands executed per day; both upload separately with ewx-cli config upload_dashboards. On the Master Data dashboard, the switched-off widgets now show the meters installed at switched-off metering points and count only switch-offs that are still current, the orphan meters widgets no longer report meters that are still linked through a metering point tag, and the non-communicating meters widgets state that customer_agreement is expected on the metering point.

📡 Integration Component​

  • SOAP hub acknowledgements: Rejection acknowledgements now take their Reason/text from the error_msg of the failing rule, so the per-reason-code reason_texts map is replaced by a single default_reason_text string; the old map is still read for one release and logs a deprecation warning. Internal validation errors no longer leak into outgoing business acknowledgements as rejection reasons, and acknowledgement-specific URLs in soap_hub.message.urls are preserved, so a message type needing a different endpoint for its acknowledgement routes there instead of falling back to the default. When the hub answers with a SOAP fault the request is no longer retried five times — it is deterministically rejected, so the message is acknowledged and the fault raised as an audit event and taskboard task; transient failures are still retried.
  • Per-message-type SOAP actions: SOAP actions can now be overridden per message type, by replacing the action template string — the get action in MetaData.json, for example — with an object keyed by message type that includes a default fallback. Existing configuration files keep working unchanged, and an updated file takes effect within the hour without a redeploy.
  • SFTP delivery is more resilient: The outbound processor retries an SFTP connection once after a 5-second delay before giving up, so transient server unavailability no longer needs a manual re-ingest; permanent authentication errors are not retried. A file that still fails to arrive now always raises a task on the configured board, including for message types generated by processing rules that carry no message ID — previously those failures were silently discarded. On the inbound side, a long-lived marker guarantees a forwarded file is never forwarded twice: if moving it to the archive folder fails, only the archive step retries, instead of the file being pulled and ingested again on every polling cycle. The deduplication TTLs are configurable via SFTP_FORWARDED_MARKER_TTL_SECONDS (default 7 days) and SFTP_PROCESSING_TTL_SECONDS (default 1 hour).
  • Simpler integration configuration: The inbound_file_tags configuration is no longer required — when it is not set, files are stored with the default MessageType and MessageStatus tags, so placeholder values are no longer needed in message maps. The obsolete INBOUND_SCHEDULE service setting has been removed; inbound polling schedules for SOAP and SFTP live in schedule_config.json.
  • RIBS market-data collection no longer loses data: When storing fetched market data in Cloud Storage fails, the message is retried and dead-lettered after repeated failures instead of being dropped silently. Outbound HTTP requests also carry a timeout, so an unresponsive external endpoint can no longer stall data collection.

26.09.01​

🖥️ UI Improvements​

  • Task Boards list keeps rows compact: In the new UI, long task board descriptions are now cut off after two lines with an ellipsis instead of stretching the row. Hover the description to read it in full, or open the task board to see it on its detail page.
  • Task datasources are clickable again: In the new UI's Task Details, datasource subjects now link to the datasource, as they did in the old UI. Click the id to open the datasource page. Subject labels also show the type alone, such as datasource or startDate, instead of the role-prefixed key.
  • Flow design rule settings: Editing a rule parameter no longer re-fetches the rule and flashes a loading spinner on every change; the rule definition is now loaded once when the settings panel opens.

⚙️ Custom Business Logic​

  • Invalid timezone no longer orphans ingested data: When a transformation config maps the datasource timezone to an invalid value, ingestion now fails cleanly at parse time and names the offending value, instead of writing the timeseries and then silently rejecting the datasource metadata write, which left orphan data behind a false "Created datasource" audit event.

🔐 API & Permissions​

  • Breaking — tag property types now use value_type: On API v1, a tag property's value type is carried by value_type instead of type, on both the request and the response, for datasource create and the tag add/update/remove endpoints. Previously the request read only type, so integrations sending the type in value_type had every non-string type (int, float, bool, ts, date, link, datasource) silently stored as string. Integrations that send or read type on a tag property must be updated: a request without value_type is rejected, and responses no longer contain type. Channel properties, timeslice-group properties and transformation-configuration mappings are unaffected.
  • Clearer error for unreadable files: Reading a stored file's contents, which is how the transformation configuration screens detect the columns in a file, failed with a raw text-encoding error for anything that was not plain UTF-8 text. A spreadsheet, archive or other binary file now reports what kind of file it is and suggests either uploading the sheet as CSV or configuring it through the API, and a CSV saved in a Windows or Western European encoding is read correctly instead of being rejected.

🚀 Notable Backend Changes​

  • Saving flows with select or multi-select rule parameters: Saving a flow with single-select or multi-select rule parameters no longer fails with a validation error, and selected parameter values are saved correctly.

📡 Integration Component​

  • Market-hub certificate loading fix: Fixed an issue where the SOAP-hub services did not always load the configured certificates, causing messages to the market hub to fail with a certificate error.

26.09.03​

🖥️ UI Improvements​

  • Change rule parameters when starting a flow: The Start flow dialog in the new console, opened from the datasource list, a datasource or a flow design, now has a Change parameters option, as the original console does. You can change a rule's parameters for this run only, or save the changes to the flow design. The option requires permission to update run configurations.

⚙️ Custom Business Logic​

  • WSDL streaming ingest fix: Files ingested through a WSDL (SOAP) streaming ingest are processed again. Since release 26.08 these files failed with an error before any data was read.

🚀 Notable Backend Changes​

  • Sign-in stays available during cloud capacity shortages: When the preferred machine type ran out of stock in a cloud region, the platform's session store could be placed on a machine type that cannot attach its disk, and it never finished starting — which made sign-in fail until an operator intervened. Services with such disks are now only placed on compatible machine types and wait for capacity instead, so they recover on their own once it returns.