SDMX and statistical management solutions for official statistics and business

SDMX is designed for statistics.
SDMX provides a structured and standardised way to describe statistical data in any domain, and a common language for exchanging both the data and its structural metadata between organisations and systems (information about how the data is or should be structured).

It allows statisticians to robustly describe the structure and rules for complex datasets, provides data providers with a clear description about what the data should look like, and data users with an unambiguous understanding of how to interpret it.

SDMX is a proven, open, non-proprietary standard sponsored by seven international organisations including the World Bank, the International Monetary Fund and the United Nations Statistics Division. It's also an International Standard ISO 17369:2013.

SDMX-ML v1.0, v2.0 and v2.1
Data sub-formats: v2.0 Generic, v2.0 Compact, v2.1 Generic, v2.1 Structure Specific
SDMX-EDI
SDMX-JSON Data 1.0 - partially compliant, full compliance planned for 2019

SDMX 3.0 will be supported when it is released.

The short answer is that the Enterprise Edition collects, stores, integrates, processes and disseminates data, the Community Edition does not (other than some specific data processing tasks described below).
Here's a summary of the main features:

• Data modelling. Create a model describing the structure of your data, rules for validating and transforming it, and (for use cases where data is collected from other organisations) the rules governing which series each data provider is expected to supply.
• Publishing structures. Make the SDMX structures which describe your data model available to people and systems as files or through the SDMX REST API.
• Data processing...
• Structural validation of data. Validate that SDMX data complies with the data model and any consistency rules that have been defined.
• Structural transformation of data. Transform datasets from one structure to another including changing the dimensionality and translating codes from one scheme to another.
• Conversion of SDMX data between versions and formats. Convert SDMX data between the XML, EDI and JSON formats, and between the v1.0, v2.0 and v2.1 versions.

• Data modelling. Create a model describing the structure of your data, rules for validating and transforming it, and (for use cases where data is collected from other organisations) the rules governing which series each data provider is expected to supply.
• Publishing structures. Make the SDMX structures which describe your data model available to people and systems as files or through the SDMX REST API.
• Data collection. Collect data from multiple provider organisations, agencies, departments or systems.
• Data storage. Load data in various formats into Fusion Registry managed local storage. Storage options include traditional databases such as Oracle and SQL Server for large datasets, and a SDMX optimised in-memory database where high performance is required.
• Data integration...
Dynamically integrate data from any number of remote providers and sources, Fusion Registry managed data stores and existing databases into a single virtual statistical data repository suitable for analytics and dissemination:
• On-demand extraction and dynamic integration of data from existing SQL databases.
• Dynamic integration of data from SDMX data web sources i.e. data retrievable as static files or from SDMX REST web services through URLs.
• Integration of series from multiple sources into the same dataset.
• Virtual data repository. Provides a single unified view of all registered datasets.
• Data processing...
• Structural validation of data. Validate that SDMX data complies with the data model and any consistency rules that have been defined.
• Structural transformation of data. Transform datasets from one structure to another including changing the dimensionality and translating codes from one scheme to another.
• Conversion of SDMX data between versions and formats. Convert SDMX data between the XML, EDI and JSON formats, and between the v1.0, v2.0 and v2.1 versions.
• Conversion of Excel and CSV data to and from SDMX.
• Perform on-the-fly time-series calculations when querying for data including arithmetic and frequency conversion.
• Data dissemination. Publish data for public or restricted access.
• Reference metadata. Enrich your data with additional, human-readable information.

Fusion Registry 9 Enterprise Edition consists of a scalable, enterprise-grade suite of components and modules centered on the Fusion Registry Enterprise Core - the main metadata registry, and data integration and processing engine.

Here's what's included:

Fusion Registry Enterprise Core plus:

Fusion Edge Server - a separate light-weight REST API server that can be deployed in scalable clusters for driving dissemination services.
Fusion Reporting Node - a separate module that acts as an interface for collecting data from external providers and sources.
Fusion Command Line Client - a command line interface suitable for scripting and automating data and metadata processes.
IM-JS - a JavaScript library providing a client-side API suitable for building Fusion Registry driven web applications like data portals.
FusionXL - a Microsoft Excel addin allowing data to be selected and downloaded directly into worksheets, authoring of data, plus browsing and maintenance of structures.
Fusion Security - a simple user authentication directory server that can be used as an alternative to Microsoft Active Directory.

Yes
Fusion Registry Enterprise Edition is time series database that uses SDMX as its underlying data model rather than relational tables or a proprietary structure.

Observation data are ingested, stored, processed, queried and output as structured, multi-dimensional time series. Datasets can consist of hundreds of millions of series and billions of observations. Fusion Registry provides a range of time series calculation functions from scalar and vector arithmetic to frequency conversion and seasonal adjustment. Data revisions (sometimes called vintages) are supported allowing the history of changes to a series' observations over time to be retained.

Fusion Registry Community Edition - the metadata registry and data processing engine.
FusionXL - a Microsoft Excel addin allowing browsing and maintenance of structures.
Fusion Security - a simple user authentication directory server that can be used as an alternative to Microsoft Active Directory.

The Community Edition is suitable for building, maintaining and publishing a formal model of your statistical data. If you're familiar with SQL databases, its equivalent to defining the tables in a schema and the values acceptable for each column, but without actually populating those tables with data.

This is useful for organisations who want a controlled, authoritative and queryable definiton of their data model to drive data collection, production and management processes. For instance, organisations who provide data can consult the Registry for a definitive description of how that data should be structured. This is the use case employed by the SDMX Global Registry which gives organisations charged with providing data on subjects like European National Accounts to Eurostat with the information they need about how it should be structured.

Register here to download.

Fusion Registry Enterprise Edition is available on annual subscription which provides license, updates, upgrades and support for a single yearly fee. The price is dependent on several factors including the number of production environments, the number of data sources or data providers, the level of support required and any use case specific configuration needed.
Please contact us to discuss your requirements.

Structural Metadata is information about how the data should be structured in terms of its dimensionality, the concepts being measured and how they are classified.
SDMX provides a range of 'structural metadata artefacts' including Concept (e.g. frequency, country, gender), Codelist (classifications) and Data Structure Definition (describing the dimensionality of a particular dataset).
Statisticians use structural metadata artefacts to define their statistical data model in Fusion Registry in the same way that a SQL database administrator will use tables, columns and keys to define the schema for a relational database.
While a Data Structure Definition (DSD) sets out the structure or dimensionality of a dataset, a Dataflow describes a specific instance. For instance, a DSD for National Accounts could be used for GDP, Central Government Debt and CPI Dataflows - different datasets that all share the same dimensionality. Data consumers retrieve the data they are interested in by querying the relevant Dataflow. In some ways, Dataflows are equivalent to Tables in the SQL relational model.
Structural metadata is also used to describe rules to be applied when collecting and processing data. In particular, Content Constraints set what data a provider is authorised to report.

Reference Metadata (sometimes called Referential Metadata) is additional human-readable information that can be attached to a dataset for the benefit of users of the data. This typically provides background information on the dataset, details about how the data was collected and the quality. Reference Metadata can be attached at the dataset level, the series level or even to individual observations. The metadata can be plain text, HTML or links to other documents and resources such as videos and images.

Time Series: Yes
Fusion Registry 9 is specifically designed for time series, and in some ways the Enterprise Edition can be thought of as a 'virtual time series database'. There is some support for executing calculations on time series to create new persistent series, or when querying for data. Examples include time series arithmetic (e.g. adding time series together), and frequency / seasonal adjustments. More time series calculations are planned for future releases.

Non Time Series: Partially, but better support is planned for later releases
Non-time series data can be managed by current releases of Fusion Registry Enterprise Edition, but every dataset must have a time dimension.

No: only aggregated data at present.
Fusion Registry Enterprise Edition currently supports only aggregated data although micro-data support is planned for future releases.

No: not at present.
VTL 2.0 support is planned for release in 2020.

Registry Content: Yes
Registry content (names, labels, descriptions etc) can be loaded, displayed and retrieved in multiple languages, including those written right-to-left such as Arabic and Hebrew. Users can choose their preferred language - content will be displayed in that language if available, or the default language otherwise. The default is often English, but it doesn't need to be.

Web User Interface: No
Text on Fusion Registry's web user interface - including the menu bar, buttons and informational messages displayed for various reasons - are always in English.

Fusion Registry is a Java enterprise web application so it will run on any computing platform with a Java run-time environment and a suitable Java web application server such as Apache Tomcat. It has a web user interface so users require only a standard web browser.

It's also designed to be very scalable, so can be run on most computers and servers with a minimum of 4GB of memory for the Community Edition. 8GB is recommended for the Enterprise Edition. More memory and processing capacity will be required depending on the volume of data and usage. There's more on this in the What are the system requirements? section.

Fusion Registry provides a comprehensive REST web services API for integration with other systems. The API is compliant with the standard SDMX RESTful web services specification.

The Community Edition REST API supports the following key functions:

• Query for structures - retrieve selected SDMX structures
• Submit metadata content - add new structures to the metadata registry or update existing ones such as adding codes to a code list
• Delete metadata content - delete selected structures from the metadata registry
• Query for schemas - retrieve XML schemas for selected structures
• Data conversion - conversion of data between SDMX formats and versions
• Data validation - validate the structure of data for compliance with the data model

The Enterprise Edition provides additional functions covering loading data, registering data sources and querying for data, plus reference metadata including:

• Query for data - retrieve data for a specified selection of series from a dataset
• Submit data content - load data into local Fusion Registry managed data stores, including the in-memory database
• Delete data content - delete selected datasets, series and observations
• Rollback data change - rollback data submissions
• Data registration - registration of external data sources
• Data reporting forms - generate Excel forms for reporting data
• Query for reference metadata - retrieve reference metadata
• Freetext search - search the metadata and data content

Yes
Metadata Technology can deliver both the Community and Enterprise Editions as fully managed cloud services. Please contact us to discuss your requirements.

Yes

Central Banks, International Organisations, National Statistics Offices and public bodies - we can provide a pre-configured cloud service, or a copy of the software distribution for on-premise evaluation. Just let us know what you want to achieve so that we can advise on the best approach.

Commercial companies, system integrators and other organisations - please contact us to discuss your requirements and plans.

On-demand support is included as part of the Enterprise Edition subscription.

The standard support package includes:

• On-demand correction of errors with target response and resolution times depending on the severity of the issue
• Minor enhancements to help support your specific use cases
• General advice as required on SDMX, integration and deployment of the software
• Regular software releases with bug-fixes and new features

Yes: pay-for support plans are available.
Fusion Registry Community Edition is free to use, but without support.
If you are planning to use the Community Edition in production we can provide on-demand break-fix support contracts with SLA response times. Please contact us to discuss your requirements.

Operating System - Windows or Linux. Other platforms including Apple Mac, HP-UX and IBM AIX should also work provided they have a suitable Java Runtime Environment (JRE) or Java Development Kit (JDK), and Java Web Application Server.

Java Runtime Environment - Java 8 and Java 10 are supported.

Java Web Application Server - Apache Tomcat 8.0, 8.5 or 9.0 are recommended. Jetty is also tested and proven. Other application servers like IBM WebSphere Application Server should also work.

Processor - A minimum of 2 cores for the Community Edition, and 4 cores for the Enterprise Edition.

Memory- 4GB is sufficient for the Community Edition. A minimum of 8GB is recommended for the Enterprise Edition Core. More memory will be required if heavy in-memory database use is planned. The Fusion Edge Server and Fusion Reporting Node both utilise the in-memory database so a minimum of 8GB should be provided for those deployments.

Web Server - A web server is recommended working in reverse-proxy mode for all production systems. Apache Web Server 2.4 is recommended although cloud deployment platforms such as Amazon Web Services and Microsoft Azure generally provide a reverse proxy service meaning that Apache is not required in those circumstances.

Yes
The Community Edition and Enterprise Edition use the same core SDMX engine. Upgrading a Community Edition installation to the Enterprise Edition is simply a case of stopping the Java application server, deploying the Enterprise software distribution and restarting the application server.

Several ways:

Fusion Data Browser - Fusion Registry's data explorer for browsing the available datasets, searching for and choosing series using the metadata, visualising data and downloading it in a range of formats. Try it.
FusionXL - an addin for Microsoft Excel allowing users to explore the available datasets, choose series using the metadata and retrieve the data directly into worksheets for further analysis.
Tableau - the Fusion Registry 9 connector for Tableau allows analysts to use Fusion Registry as a Tableau data source.
REST Web Services - many applications like Excel and Microsoft Power BI can take data directly from Fusion Registry's web services. There's an interactive web service query builder to help select the series of interest and generate the necessary URLs. Try it.
'R', SAS and Matlab - open-source data connectors, compatible with Fusion Registry's SDMX REST Web Services, are available for 'R', SAS and Matlab. Get the connectors.

Yes
Tableau users can connect directly to Fusion Registry and use it as a data source using the Fusion Registry Web Data Connector.

Yes
At present, there is no native connector for Microsoft Power BI to use Fusion Registry as a data source. However, there are several alternative options:

• Export data as Excel or CSV files and import into Power BI
• Use Power BI's Web connector to import CSV data directly from Fusion Registry's REST data web services

Yes
Microsoft Active Directory is supported. For organisations without Active Directory, Fusion Security can be used. Fusion Security is Metadata Technology's proprietary directory server.

No: not at present.
Single sign-on support using Microsoft Active Directory is planned for release in 2019.

Yes
The Fusion Data Browser (Fusion Registry's web app for exploring and visualising the data) can be used to select the series to visualise, design the required chart or table and generate a link or HTML code allowing it to be shared or embedded in websites. There's a range of charts to choose from including time-series (line), bar, pie, pyramid, heat map and polar, plus a full interactive pivot table.
Here's a live example:

Yes
The simplest approach is to include geographic dimensions such as 'country','reference area' or 'region' in your dataset model and to use a separate GIS, visualisation or analytics package to build and publish the required maps. Fusion Registry can export the geographical metadata together with the observation data in a range of suitable formats including Excel, CSV and JSON. Tableau users can use the Fusion Registry Web Data Connector which allows Fusion Registry to act as a direct data source.

While Fusion Data Browser (Fusion Registry's web app for exploring and analysing the data, and building embeddable visualisations) has a library of charts and tables to choose from, it doesn't currently support geographic mapping.

When designing your data model, geographical dimensions should use codes that your GIS package will understand, ISO 3166 two or three character country codes for instance. Where observations relate to precise coordinates such as lattitude and longitude, hold these as SDMX series or observation level attributes unless you intend for each unique coordinate to be a separate series.

Here's an alternative example using Google Data Studio to visualise World Bank Life Expectancy data as a map. The dashboard is live, driven directly from the Fusion Registry 9 Demo Server using Metadata Technology's Fusion Registry connector for Google Data Studio.