Message Mapper

To process all messages generically, we added a mapper to the system. This mapper creates a common message object from every message, and holds vital information on destinations, response and message process.

Note: For SFTP-based integrations, the mapping structure is simplified since SFTP files don't have SOAP headers. See the SFTP-Specific Configuration section below for details on SFTP field mapping.

Below is an example mapping with mandatory fields and explanation on what they mean and what they should hold (applicable to SOAP messages).

{
    "inbound_filename": "the name of the file that will show up in the filemanager",
    "outbound_needs_ack": "true for async false for sync",
    "header_maps": {
        "technical_message_id": "the technical_message_id from the soap:Header/MessageAddressing",
        "correlation_id": "the correlation_id from the soap:Header/MessageAddressing",
        "sender_id": "the sender_id from the soap:Header/MessageAddressing",
        "receiver_id": "the receiver_id from the soap:Header/MessageAddressing",
        "carrier_id": "the carrier_id from the soap:Header/MessageAddressing",
        "content_type": "the content_type from the soap:Header/MessageAddressing",
    },
    "field_maps": {
        "process_type_id": "id of the message type (N10, N11) of not there think of logical abbreviation of message type",
        "creation_timestamp": "message creation timestamp",
        "allocation_date": "allocation date: usually period end or if na then now",
        "----COMMENT----": "parsed_allocation_date: 'parsed allocation date to use in file tags",
        "domain": "ean/id of the metering point",
        "message_id": "unique (uuid) of the message",
        "revision_number": "follows the message_id default to 1",
        "correlation_id": "unique (uuid) same over request and response",
        "receiver_id": "ean of the receiving party",
        "receiver_contact_type": "contact type of the receiving party",
        "sender_id": "ean of the sending party",
        "sender_contact_type": "contact type of the sending party",
        "-----COMMENT----": "conditional fields for async messages",
        "conversation_id": "ONLY FOR soap_hub.name = 'mmchub' unique (uuid) of the conversation can be ''",
        "new_ack_message_id": "ONLY FOR soap_hub.name = 'mmchub' unique (uuid)",
        "new_correlation_id": "ONLY IF GLDPM = True: unique (uuid)",
        "document_type": "ONLY FOR soap_hub.service_name = 'gldpm': type of the received GL_MarketDocument (e.g. A67); emitted as received_MarketDocument.type in the acknowledgement",
        "reason_code": "TAKEN FROM ACKS IN ASYNC PROCESS reason/rejection code",
        "reason_text": "TAKEN FROM ACKS IN ASYNC PROCESS reason/rejection text",
        "ack_created_datetime": "TAKEN FROM ACKS IN ASYNC PROCESS acknowledgement creation timestamp",
        "------COMMENT----": "conditional fields for sync messages",
        "external_ref": "FOR PUD ACK: external reference",
        "external_mrid": "FOR PUD ACK: external mrid",
        "dossier_id": "FOR PUD ACK: dossier id"
    },
    "inbound_file_tags": {
        "----COMMENT----": "OPTIONAL: tags to be used in the fileservice. tagname: ref to name in field_maps. Omit the whole section when no custom tags are needed",
        "SenderID": "sender_id",
        "-----COMMENT----": "...",
        "AllocationPoint": "domain"
    },
    "ack_maps": {
        "msg_type": {
            "msg_type": "ONLY if different from default. multiple possible",
            "ns": "namespace of the acknowledgement",
            "default_reason_code": "if positive acknowledgement"
        },
        "default": {
            "msg_type": "Acknowledgement or response message type",
            "ns": "namespace of the acknowledgement",
            "default_reason_code": "if positive acknowledgement 000",
            "default_reason_text": "OPTIONAL Reason/text for a positive acknowledgement, e.g. Message fully accepted. Mandatory for GLDPM per the dependency table."
        }
    },
    "soap_hub": {
        "name": "mmchub or edsn",
        "service_name": "mmchub, gldpm, PUD or CAR",
        "base_url": {
            "test": "base url to test environment",
            "production": "base url to production environment"
        },
        "message": {
            "multi_response": {
                "multi_response": "False/True if multiple responses are expected (loop until empty response)",
                "check_field": "what field to check on the response (point to field in field_maps of response mapper) if the field is not empty the response is not empty and a new request will be sent",
                "message_id_element": "element of the message id to regenerate on re-send",
                "creation_ts_element": "element of the CreationTimestamp to regenerate on re-send"
            },
            "urls": {
                "msg_type": "{base_url}<URLPART>/{msg_type}/<URLPART> ONLY if different from default. multiple possible",
                "default": "{base_url}<URLPART>/{msg_type}/<URLPART>",
                "-----COMMENT----": "conditional fields for async messages",
                "get": "{base_url}url",
                "list": "{base_url}url"
            },
            "soap_actions": {
                "-----COMMENT----": "each action is either a template string applied to all message types, or an object keyed by message type with a 'default' fallback",
                "send_action": "SOAP Send action",
                "-----COMMENT----": "conditional fields for async messages",
                "get_action": {
                    "default": "SOAP Get action",
                    "SomeMessageType": "SOAP Get action override for SomeMessageType"
                },
                "list_action": "SOAP List action"
            }
        },
        "acknowledgements":{
        	"-----COMMENT----": "acknowledgements can be used if soapActions and URL's for acknowledgements differ from the messages themselves.",
            "urls": {
                "default": "{base_url}<URLPART>/{ack_type}/<URLPART>",
                "get": "{base_url}url",
                "list": "{base_url}url"
            },
            "soap_actions": {
                "send_action": "SOAP Send action",
                "get_action": "SOAP Get action"
            }
        }
    }
}

The mapper files should have the same name as the message_type and are uploaded to the config bucket in the folder “maps”

SOAP Actions: Which File to Change

The three SOAP actions (send_action, get_action, list_action) are not all read from the same file. This is one of the most common sources of misconfiguration. The table below shows exactly which GCS file controls each action:

Action	Controlling file	Scope	Notes
`send_action`	`maps/{MessageType}.json`	Per message type	Change here to update the outbound send action for a specific message type
`get_action`	`maps/MetaData.json`	All message types	A template string — `{msg_type}` is substituted at runtime with the actual message type name. Supports per-message-type overrides (see below). Changing `get_action` in a per-message-type file has no effect.
`list_action`	`maps/ListMap.json`	All message types	Applies to all list calls regardless of message type. Supports per-message-type overrides (see below). Changing `list_action` in a per-message-type file has no effect.
`acknowledgements.soap_actions.send_action`	`maps/{MessageType}.json`	Per message type	Only needed when the acknowledgement endpoint differs from the message endpoint
`acknowledgements.soap_actions.get_action`	`maps/{MessageType}.json`	Per message type	Only needed when the acknowledgement get action differs from `MetaData.json`

When to change which file

Changing the send action for a specific message type: Update soap_hub.message.soap_actions.send_action in maps/{MessageType}.json.

Changing the get action (used when the scheduler retrieves a message from the hub): Update soap_hub.message.soap_actions.get_action in maps/MetaData.json. This value is a template string, e.g.:

http://sys.svc.tennet.nl/MMCHub/get{msg_type}

The {msg_type} placeholder is replaced at runtime with the actual message type name.

Overriding an action for a specific message type: Every action (send_action, get_action, list_action) also accepts an object keyed by message type, with a default entry as fallback. Use this when one message type does not follow the common action pattern — for example when the hub introduces a versioned action for a single type:

"soap_actions": {
    "get_action": {
        "default": "http://sys.svc.tennet.nl/MMCHub/get{msg_type}",
        "AggregatedVolumeSettlementResultSeriesNotification": "http://sys.svc.tennet.nl/MMCHub/get{msg_type}-v2.0"
    }
}

The lookup key is the message type being sent/retrieved; when it is not present, the default template is used. The plain-string form remains fully supported (it behaves exactly like an object containing only default), so existing configuration files keep working unchanged. Because the get and list actions live in MetaData.json / ListMap.json, per-message-type get/list overrides go into those files, not into the per-message-type mapping file. Note that configuration is cached — an updated file takes effect after the config cache TTL expires (up to one hour), without a redeploy.

Changing the list action (used when the scheduler or broker lists available messages on the hub): Update soap_hub.message.soap_actions.list_action in maps/ListMap.json.

caution

The get_action and list_action fields that appear in per-message-type mapping files (e.g. maps/SeriesNotification.json) are not used for the actual get/list operations. Those fields in per-message-type files have no effect on the scheduler or broker behaviour. Only the values in MetaData.json and ListMap.json are read for these operations.

URL resolution for acknowledgements

When sending an acknowledgement, the service looks up the URL from soap_hub.message.urls using the ack message type as the key, falling back to default if the key is not present.

All keys defined in message.urls are preserved at runtime. This means you can add an explicit entry for the ack type alongside the inbound notification type:

"urls": {
    "AggregatedSettlementResultSeriesNotification": "{base_url}Measurements_Allocations/AggregatedSettlementResultSeriesNotification/v1.0",
    "AggregatedSettlementResultSeriesAcknowledgement": "{base_url}Measurements_Allocations/AggregatedSettlementResultSeriesAcknowledgement/v1.0",
    "default": "{base_url}Measurements_Allocations/{msg_type}/v2.0"
}

In this example, when sending the acknowledgement the service finds AggregatedSettlementResultSeriesAcknowledgement in the urls dict and uses v1.0. Without the explicit key, it would fall back to default (v2.0), which may be rejected by the hub.

When to use acknowledgements block instead: Use soap_hub.acknowledgements only when the acknowledgement requires both a different URL and different SOAP actions from the inbound message (currently only applicable to GLDPM). For all other cases, add the ack type key directly to message.urls.

Header maps

Header maps hold the mapping from the soap header of a message to the common structure. They define a function to use and the XPath to the actual element in the message.

"header_maps": {
        "common_name": {
            "function": "get_from_message",
            "args": "xPath"
        },

The common_name holds the key of the field, like described below. See Field maps for functions and extra explanation.

Field maps

Field maps hold the mapping from the message to the common structure. They define a function to use and the XPath to the actual element in the message.

"field_maps": {
        "common_name": {
            "function": "get_from_message",
            "args": "xPath"
        },

The common_name holds the key of the field, like described below.

We then have a couple of functions that can be performed.

"get_from_message" This gets the element value from a message based on the XPath defined in the "args" field

"creation_timestamp": {
    "function": "get_from_message",
    "args": "CreationTimestamp"
},

"return_string" This returns the exact string that is defined in the "args" field. This is used to set a default value

"string": {
    "function": "return_string",
    "args": "hallo"
},

"parse_to_date" This parses a timestamp from the xPath defined in the "args" field to a simple date

"timestamp": {
    "function": "parse_to_date",
    "args": "CreationTimestamp"
},

"current_date_time" This returns the current timestamp"args" field should be empty.

"timestamp": {
    "function": "current_date_time",
    "args": ""
},

"generate_uuid" This returns a new uuid4 "args" field should be empty.

"new_id": {
    "function": "generate_uuid",
    "args": ""
},

"not_null_value" If the first function returns nothing, perform the second one. "args" holds the functions

"domain": {
    "function": "not_null_value",
    "args": [
        {
            "function": "get_from_message",
            "args": "MarketEvaluationPoint/mRID"
        },
        {
            "function": "get_from_message",
            "args": "Domain/mRID"
        }
    ]
},

"concat_hyphen" this adds two fields together with a -

"inbound_filename":{
    "function": "concat_hyphen",
        "args": [
            {
                "function": "get_from_message",
                "args": "ProcessTypeID"
            },
            {
                "function": "get_from_message",
                "args": "MessageID"
            }
        ]
},

Acknowledgement maps

ack_maps defines how the acknowledgement is built per acknowledgement msg_type (with a default fallback):

msg_type: the acknowledgement/response message type element name.
ns: the namespace of the acknowledgement document.
default_reason_code: the Reason/code used for a positive acknowledgement (overridable per message).
default_reason_text (optional): the Reason/text emitted for a positive acknowledgement (paired with default_reason_code). Rejection texts are not configured here — they come from the error_msg of the failing rule in the rules config (see Validation Rules). When empty, no text element is added. (Deprecated: the former reason_texts map is still read for one release, but only its entry for default_reason_code.)

GLDPM acknowledgement (`soap_hub.service_name = "gldpm"`)

The IEC 62325-451-1 Acknowledgement_MarketDocument requires two elements that are optional in the XSD but mandatory per the market's semantic-validation dependency table:

received_MarketDocument.type — populated from the document_type field map (the type of the received GL_MarketDocument, e.g. A67). Emitted between received_MarketDocument.revisionNumber and received_MarketDocument.createdDateTime.
Reason/text — for a positive acknowledgement, populated from ack_maps.default_reason_text (e.g. A01 = "Message fully accepted"); for a rejection, populated from the error_msg of the failing rule in the rules config, or by the validation logic itself for dynamic texts (e.g. 999 per-validation TEN-xxxxxx: ... strings, A51 revision conflicts).

Both are conditional: without document_type / a reason text the elements are simply not emitted, so the configuration must provide them for GLDPM to satisfy the dependency table. Reason texts have a single source each — the positive text lives once in ack_maps.default_reason_text, rejection texts live on their rules in the rules config — so there is no per-code text map to keep in sync.

SFTP-Specific Field Mapping

For SFTP-based integrations, the mapping structure is simplified since SFTP files don't have SOAP envelopes or headers. The configuration is stored in field_map_configs/<message_type>.json and only contains a field_maps section.

SFTP Field Maps Structure

{
    "field_maps": {
        "inbound_filename": {
            "function": "concat_hyphen",
            "args": [
                {"function": "return_string", "args": "dre"},
                {"function": "current_date_time", "args": ""}
            ]
        },
        "domain": {
            "function": "return_string",
            "args": "domain"
        },
        "message_id": {
            "function": "concat_hyphen",
            "args": [
                {"function": "return_string", "args": "dre"},
                {"function": "current_date_time", "args": ""}
            ]
        },
        "sender_id": {
            "function": "return_string",
            "args": "Acme"
        },
        "creation_timestamp": {
            "function": "current_date_time",
            "args": ""
        },
        "receiver_id": {
            "function": "return_string",
            "args": "Energyworx"
        }
    }
}

Key Differences for SFTP Mapping

No Header Maps: SFTP files don't have SOAP headers, so there is no header_maps section
No SOAP Hub Configuration: The soap_hub section is not applicable
No Acknowledgement Maps: SFTP is file-based and doesn't use acknowledgement messages like SOAP
Simplified Field Maps: Often use return_string and current_date_time functions rather than extracting from message structure
File Tags Stored Separately: File tags are configured in a separate file file_tags/<message_type>.json
Response Config Stored Separately: Response configurations including archive locations are in response_configs/<message_type>.json

Supported Functions for SFTP

All the standard mapping functions work with SFTP messages:

get_from_message - Extract values from XML content (if the file is XML)
return_string - Return static/default values
current_date_time - Generate current timestamp
generate_uuid - Generate unique identifiers
concat_hyphen - Concatenate multiple values
not_null_value - Fallback logic
parse_to_date - Parse timestamps to dates

The main difference is that SFTP often relies more heavily on generated values (return_string, current_date_time, generate_uuid) rather than extracting from SOAP headers, since there are no SOAP headers in SFTP files.

For complete example configuration files (field maps, file tags, response configs, and connections), see the Configuration Reference.

SOAP Actions: Which File to Change​

When to change which file​

URL resolution for acknowledgements​

Header maps​

Field maps​

Acknowledgement maps​

GLDPM acknowledgement (soap_hub.service_name = "gldpm")​

SFTP-Specific Field Mapping​

SFTP Field Maps Structure​

Key Differences for SFTP Mapping​

Supported Functions for SFTP​