Skip to main content

Datasources

Search Datasources​

When you create a datasource using the Energyworx API, there are important considerations about when and how it becomes searchable:

Datasources with Tags (Immediately Searchable)​

If you create a datasource with at least one tag, it will be immediately available in the search interface. The platform automatically indexes datasources with tags to the search engine, making them instantly discoverable.

Example:

{
  "id": "METER_001",
  "classifier": "POWER",
  "tags": [
    {
      "tag": "location",
      "properties": [
        {"key": "building", "value": "Building A"}
      ]
    }
  ]
}

✅ This datasource will appear in search results right away.

Datasources without Tags (Delayed Search Availability)​

If you create a datasource without any tags, it will not be immediately searchable. These datasources only become available in search during one of the following events:

  1. When you add tags to the datasource - Adding tags automatically refreshes search results
  2. Periodic maintenance - Search results are updated during scheduled maintenance procedures

Example:

{
  "id": "METER_002",
  "classifier": "POWER",
  "tags": []
}

⚠️ This datasource will not appear in search results until tags are added or bulk indexing occurs.

Best Practices​

To ensure your datasources are immediately discoverable:

  • Always include at least one tag when creating datasources via API
  • Use meaningful tags that help categorize and identify your datasources
  • Common tags include: location, device_type, customer, status, etc.
  • If you need a datasource without specific business tags, consider adding a source or created_via tag to ensure searchability

Recommended approach:

{
  "tags": [
    {
      "tag": "source",
      "properties": [
        {"key": "created_via", "value": "api"}
      ]
    }
  ]
}

This ensures your datasource is indexed immediately while providing useful metadata about its origin.

Creating and Updating Datasources​

Append-Only Design Pattern

The Energyworx platform follows an append-only design pattern for resources with built-in versioning, including datasources, tags, and datapoints. This design ensures data integrity, maintains complete audit trails, and supports regulatory compliance requirements.

How Datasource Creation Works​

When you create a datasource using the API (e.g., /ewx/v1/datasource/create), the platform handles both creation and updates through the same endpoint:

  • New Datasource: If the datasource ID does not exist, a new datasource is created with the provided properties
  • Existing Datasource: If the datasource ID already exists, the datasource properties are updated with the new values

This behavior is consistent with the platform's append-only architecture, where resources with built-in versioning support updating properties through creation endpoints rather than having separate update endpoints.

Important Considerations​

No Duplicate Prevention: The API does not prevent you from calling the create endpoint with an existing datasource ID. Instead, it will update the existing datasource's properties. This means:

  • You will receive a successful response (200 OK) even when updating an existing datasource
  • The endpoint name "create" may be misleading, as it also handles updates
  • There is no separate dedicated update endpoint for datasources in API v1.5

Best Practices:

  • Be aware that calling the create endpoint multiple times with the same ID will update the datasource each time
  • If you need to verify whether a datasource exists before creating it, use the get endpoint first
  • For production workflows, ensure your integration logic handles this idempotent behavior appropriately
  • Consider implementing client-side checks if you need to distinguish between create and update operations

Re-ingestion Workflows: This behavior supports re-ingestion scenarios where datasources need to be recreated or refreshed. The platform allows you to send the same datasource data multiple times without errors, updating properties as needed.

Example​

// First call - creates a new datasource
POST /ewx/v1/datasource/create
{
  "id": "METER_001",
  "classifier": "POWER",
  "description": "Main building meter"
}
// Response: 200 OK - Datasource created

// Second call with same ID - updates the existing datasource
POST /ewx/v1/datasource/create
{
  "id": "METER_001",
  "classifier": "POWER",
  "description": "Updated description for main building meter"
}
// Response: 200 OK - Datasource properties updated

Datasource Deletion Policy​

Append-Only Platform Design

The Energyworx platform is designed as an append-only system for auditing purposes. This means that datasource deletion is not available through the standard platform interface or API. This design ensures data integrity, maintains complete audit trails, and supports regulatory compliance requirements.

Deletion in Non-Production Environments​

For development and testing environments only, Energyworx provides a datasource deletion service on a best-effort basis. If you need to delete datasources from a non-production environment:

  1. Submit a request through the Energyworx Service Desk
  2. Provide the following information:
    • Environment name (e.g., ewx-dev-001, ewx-staging-002)
    • Namespace ID
    • List of datasource IDs to be deleted

Important Considerations:

  • Best-Effort Service: While we make every effort to remove all related data, some references to deleted datasources may persist in:

    • Audit events
    • Search indices
    • Flow execution history
    • Other platform objects that reference the datasources

  • Impact on Platform Experience: Lingering references may affect functionality in:

    • Search results
    • Dashboards
    • Reports
    • Historical flow executions

  • Production Environments: Datasource deletion is not recommended and typically not permitted in production environments due to the platform's design principles.

Alternative: Deactivating Datasources​

Instead of deletion, consider deactivating or marking datasources as inactive:

  • Stop ingesting new data for the datasource
  • Update datasource tags to indicate inactive status
  • Use datasource filters in searches and reports to exclude inactive datasources

This approach maintains the audit trail while reducing clutter in active workflows.

Multiple Datasources View​

This article describes how to create a graph with timeseries data from multiple datasources.

  1. Go to Search Datasources.
  2. Search and select the datasources that you want to view.
  3. Click on “Create view from selection” in the upper-right.
  4. Select the date range you want to see.
  5. Click “View Datasources”, you will be directed to a view with multiple datasources.
  6. Scroll down to the chart, and select data to be plotted.

See below a video for this.