Skip to main content

File Management

File Management is the console's view on the file manager: the storage behind your namespace where every file that has been uploaded to, or generated by, the platform ends up. From here you upload new files, tag them, hand them to a Market Adapter for ingestion, download them again, and clean up.

Open it under Smart Integration → File Management. The page needs the read storage permission; uploading and ingesting need create storage, tagging needs update storage and deleting needs delete storage (see IAM and Permissions).

info

This page covers the console. For getting files in without a person at the keyboard (a write-only service account dropping files straight into storage) see Data Ingestion → Automated delivery.

Finding files​

A namespace can hold tens of thousands of files, so the list is loaded a page at a time and all filters run on the server:

  • Search by file name in the search box and press Enter.
  • Market adapter — type to filter the adapter list, then pick one to show only the files it is assigned to.
  • Tag — files carrying the given tag.
  • Created between — a start and end date.

Click Search to apply, Clear to reset. Scroll down to load more results; the counter above the table shows how many files match.

Right-click a value in the table (a market adapter, a tag, a date) for shortcuts: copy it, or filter the list on it.

The file list​

ColumnWhat it shows
File nameClick to download the file. A lock icon marks a read-only file (generated by the platform).
SizeFile size.
CreatedWhen the file was stored.
Last updatedLast change to the file or its tags.
Market adapterThe adapter that parses the file on ingestion, if one is assigned.
TagsTags on the file; when they don't fit, a +N pill opens the details panel with the full list.
Uploaded byThe user (or service account) that stored the file.

Each row has two actions: details (the eye icon) opens the file's side panel, ingest (the play icon) starts ingestion of that one file.

File details​

The details panel shows the file's identifier, size, creation and update times, uploader and market adapter. From it you can:

  • add or remove tags (read-only files can't be changed);
  • download the file;
  • create a Transformation Configuration based on this file. The file is used as the sample to map columns and payload types from, which is the quickest way to start a new Transformation Configuration.

Uploading files​

Click Upload (top right). In the dialog:

  1. Drag one or more files onto the drop zone, or click it to browse (CSV, Excel, XML, JSON, …).
  2. Optionally add tags. Type a tag and press Enter; tags are attached to every file in the upload.
  3. Optionally select the Market Adapter that should parse the files.
  4. Turn on Ingest after upload to start ingestion as soon as the files are stored. This needs a market adapter; without one the files are stored only.

The dialog reports per-file progress. Files that failed to upload can be retried from the same dialog.

This is also the easiest way to try out a new Market Adapter or Transformation Configuration before automating file delivery.

Working with several files at once​

Tick the checkboxes to select files. A floating action bar appears with:

  • Add tags / Remove tags — change tags on all selected writable files.
  • Assign Market Adapter — set (or, with Override Market Adapter, replace) the adapter on the selection.
  • Ingest — ingest the selection. Files without an adapter need one picked in the ingest panel; all files in one ingest run use the same adapter.
  • Download — download the selected files.
  • Delete — remove the selected files after a confirmation. Read-only files are skipped.

Read-only files are excluded from every change; the panels tell you how many were skipped.

Ingestion​

Ingesting a file sends it through its Market Adapter, which parses it into datasources and timeseries. Ingestion runs in the background; follow its progress and results under Observability → Audit Events or on the datasource's Audit Events tab.

A file can be ingested more than once, for example after fixing its Transformation Configuration.

Payload type mapping​

Administrators (update admin) can open Payload Mapping from the toolbar. It maps payload types to market adapters: when a file arrives with a matching payload type (for example through automated delivery or an integration), it is ingested with the mapped adapter without anyone assigning it by hand.