Fusion Registry®10

Fusion Registry 10 is for central banks, international organisations, national statistics offices, development banks and regional statistics authorities who need to collect, integrate, store, process and publish aggregated official statistics.

Fusion Registry implements the SDMX international standard for official statistics making it robust, non-proprietary and easy to exchange both data and metadata.

SDMX is used right down to its core, so Fusion Registry is not just a database with an SDMX interface but a true aggregated statistical data warehouse designed by statisticians explicitly to meet the requirements of official statistics organisations.

Start for free

SDMX is designed for statistics.
SDMX provides a structured and standardised way to describe statistical data in any domain, and a common language for exchanging both the data and its structural metadata between organisations and systems (information about how the data is or should be structured).

It allows statisticians to robustly describe the structure and rules for complex datasets, provides data providers with a clear description about what the data should look like, and data users with an unambiguous understanding of how to interpret it.

SDMX is a proven, open, non-proprietary standard sponsored by seven international organisations including the World Bank, the International Monetary Fund and the United Nations Statistics Division. It's also an International Standard ISO 17369:2013.

SDMX-ML v1.0, v2.0 and v2.1
Data sub-formats: v2.0 Generic, v2.0 Compact, v2.1 Generic, v2.1 Structure Specific
SDMX-EDI
SDMX-JSON 1.0 (data and structures)

SDMX 3.0 will be supported when it is released.

No
All Fusion Registry variants are closed source.

However, if you're a central bank or international organisation user of the Enterprise Edition, you'll get access to the source code if you need it.

Yes

There are now two free versions: Fusion Registry Community Edition which is an SDMX metadata registry, and Fusion Registry Data Essentials which does the same as the Community Editon, plus all of the essential functions for loading, integrating, storing, processing and publishing aggregated statistical data using the SDMX standard.

The Community Edition is free for anyone to download and use. Fusion Registry Data Essentials is free for qualifying organisations.

There's more here.

Pricing information for the paid-for commercial editions can be found here.

As a 'structural metadata registry', its fundamental purpose is to act as a controlled source of metadata for other applications and processes that do handle data. For instance, an application for reporting data to another organisation like the IMF could query a Registry for up-to-date details about how the data should be structured and classified. This is an example of a 'metadata-driven process'.

The Community Edition can also be used as an interactive tool for authoring and maintaining structural metadata through its web user interface.

Structural Metadata is information about how the data should be structured in terms of its dimensionality, the concepts being measured and how they are classified.
SDMX provides a range of 'structural metadata artefacts' including Concept (e.g. frequency, country, gender), Codelist (classifications) and Data Structure Definition (describing the dimensionality of a particular dataset).
Statisticians use structural metadata artefacts to define their statistical data model in Fusion Registry in the same way that a SQL database administrator will use tables, columns and keys to define the schema for a relational database.
While a Data Structure Definition (DSD) sets out the structure or dimensionality of a dataset, a Dataflow describes a specific instance. For instance, a DSD for National Accounts could be used for GDP, Central Government Debt and CPI Dataflows - different datasets that all share the same dimensionality. Data consumers retrieve the data they are interested in by querying the relevant Dataflow. In some ways, Dataflows are equivalent to Tables in the SQL relational model.
Structural metadata is also used to describe rules to be applied when collecting and processing data. In particular, Content Constraints set what data a provider is authorised to report.

,

Reference Metadata (also called Referential Metadata) is human-readable information that can be attached to datasets, series, observations, concepts and other data and structural metadata elements. If you want to provide some explanatory notes to help users interpret a dataset, Reference Metadata is the thing to use. And it's not just restricted to text. Reference Metadata can be plain text, HTML or links to other documents and resources such as videos and images.

Time Series: Yes
Fusion Registry is designed primarily for aggregated time series.

Cross Sectional: Partial
Fusion Registry can model cross sectional data and manage it to an extent. However, many functions assume the presence of a time dimension so the best approach is to model cross sectional datasets with a time dimension, but give all observations the same survey time.

No: only aggregated data at present.
This is parimarily a limitation of the current 2.1 version of SDMX which is mainly concerned with aggregated data. Better support for microdata is planned in the forthcoming SDMX 3.0 information model including alternative measure cardinality on Data Structure Definitions, and more relaxed Codelists that don't need to comply with the current strict rules imposed by the standard.

No: not at present.
VTL 2.0 support is on the development roadmap.

Yes: using Metadata Technology's Time Series Expression Language.
Fusion Registry's time series calculation engine executes formulas in expressed in Metadata Technology's Time Series Expression Language on existing series in its statistical data repository to create new series. In this sense it follows the functional programming paradigm where the series / datasets referenced by an expression are immutable, and the result is always new series.

  • Transformation - SDMX Structure Maps can be used to transform data to different structures and classification schemes.
  • Calculation - Calculate new series using Metadata Technology's Time Series Expression Language.
  • Format Conversion - Convert data between formats including SDMX formats, non-SDMX formats like Excel.
  • Aggregation - Aggregate series to a higher level using definable functions.
  • Frequency Conversion - Convert time series to different frequencies using definable re-sampling and interpolation functions.
  • Structural Validation - Check data for compliance with the information model defined by the SDMX structural metadata. This can also check for other potential problems such as duplication observations.
  • Semantic Validation - Check data for compliance with definable business rules such as observations for individual classifications match the value reported for TOTAL.

Fusion Registry does not have a workflow engine so has no built-in way for users to define and execute processes consisting of sequences of tasks.
However, third party workflow tools can be easily integrated using Fusion Registry's rich REST API to execute tasks and receive results.
Shell scripts (.sh, .bat and Windows Powershell) can also be created using Fusion Command Line Interface allowing certain tasks such as loading data and metadata to be scripted.

No
Although all functions can be executed using the REST API, so tools like Postman or Curl can be used instead.

Registry Content: Yes
Registry content (names, labels, descriptions etc) can be loaded, displayed and retrieved in multiple languages, including those written right-to-left such as Arabic and Hebrew. Users can choose their preferred language - content will be displayed in that language if available, or the default language otherwise. The default is often English, but it doesn't need to be.

Fusion Registry Web User Interface: No
Text on Fusion Registry's web user interface - including the menu bar, buttons and informational messages displayed for various reasons - are always in English.

Fusion Data Browser Web User Interface: Yes
Legends, buttons, menu items and other text can all appear in a language of a your choice using configurable language translation JSON files.

Fusion Registry is a Java web application so it will run on any computing platform with a Java run-time environment and a suitable Java web application server such as Apache Tomcat. It has a web user interface so users require only a standard web browser.

It's also designed to be very scalable, so can be run on most computers and servers with a minimum of 4GB of memory for the Community Edition. 8GB is recommended for the Enterprise Edition. More memory and processing capacity will be required depending on the volume of data and usage. There's more on this in the What are the system requirements? section.

Fusion Registry provides a comprehensive REST web services API for integration with other systems. The API is compliant with the standard SDMX RESTful web services specification.

The Community Edition REST API supports the following key functions:

  • Query for structures - retrieve selected SDMX structures
  • Submit metadata content - add new structures to the metadata registry or update existing ones such as adding codes to a code list
  • Delete metadata content - delete selected structures from the metadata registry
  • Query for schemas - retrieve XML schemas for selected structures
  • Data conversion - conversion of data between SDMX formats and versions
  • Data validation - validate the structure of data for compliance with the data model

Data Essentials and the Enterprise Edition provides additional functions covering loading data, registering data sources and querying for data including:

  • Query for data - retrieve data for a specified selection of series from a dataset
  • Submit data content - load data into local Fusion Registry managed data stores, including the in-memory database
  • Delete data content - delete selected datasets, series and observations
  • Rollback data change - rollback data submissions
  • Data registration - registration of external data sources
  • Data reporting forms - generate Excel forms for reporting data
  • Freetext search - search the metadata and data content

The Enterprise Edition also supports:

  • Reference metadata - query for and submit reference metadata
  • Audit - query audit logs
  • Fusion Edge Server - generate data and metadata packages for publication to Fusion Edge Servers
  • Content Security - query for and manage Content Security rules

Operating System - Windows or Linux. Other platforms including Apple Mac, HP-UX and IBM AIX should also work provided they have a suitable Java Runtime Environment (JRE) or Java Development Kit (JDK), and Java Web Application Server.

Java Runtime Environment - Java 8 and Java 10 are supported.

Java Web Application Server - Apache Tomcat 8.0, 8.5 or 9.0 are recommended. Jetty is also tested and proven. Other application servers like IBM WebSphere Application Server should also work.

Processor - A minimum of 2 cores for the Community Edition, and 4 cores for the Enterprise Edition.

Memory- 4GB is sufficient for the Community Edition. A minimum of 8GB is recommended for the Enterprise Edition Core. More memory will be required if heavy in-memory database use is planned. The Fusion Edge Server and Fusion Reporting Node both utilise the in-memory database so a minimum of 8GB should be provided for those deployments.

Web Server - A web server is recommended working in reverse-proxy mode for all production systems. Apache Web Server 2.4 is recommended although cloud deployment platforms such as Amazon Web Services and Microsoft Azure generally provide a reverse proxy service meaning that Apache is not required in those circumstances.

The Community Edition and Data Essentials use the same core SDMX engine. To upgrade, simply stop the Java web application server, deploy the Data Essentials software distribution and restart the application server.

What can it do?

First and foremost, Fusion Registry is a statistical data warehouse for aggregated data which allows statisticians to build and curate a highly-structured, categorised and searchable repository of statistical datasets. That makes it suitable for many use cases where a governed source of statistical data is required. The most common are as a statistical data warehouse for internal statistics and economics teams, for reporting data to other organisations, and for public data dissemination. There's also comprehensive support for data collection use cases, particularly for organisations like regional statistics authorities who are primarily concerned with collecting and integrating data from multiple providers.

Fusion Registry uses the SDMX information model which structures statistical data as multi-dimensional cubes using a form of star schema. However, it will also act as a searchable catalogue of discrete series supporting use cases where a series-oriented approach is more intuitive. It is primarily designed for time series, but will also handle cross-sectional data like census.

Fusion Registry will seamlessly integrate loaded data held in its own storage with that from any number of local and remote data sources to form a seamless virtual repository. Instead of a language like SQL, the SDMX compliant REST API is used to query for and manipulate both data and metadata.

Fusion Registry excels at managing aggregated data making it suitable for the latter stages of the official statistics lifecycle when building, curating, exchanging and publishing highly structured statistical datasets is what's required. The goal however is a completely integrated SDMX metadata-driven processing pipeline covering the whole of the official statistics lifecycle. Check out the Fusion Registry 2020-2022 development roadmap to find out what's planned.

Free Tier


Fusion Registry Community Edition

SDMX Metadata Registry

A controlled repository for SDMX structural metadata with a web user interface for interactively browsing, authoring and maintaining structures, and an SDMX-compliant REST API.

Java application for on-premise installation on Windows or Linux.

NEW Fusion Registry Data Essentials

SDMX Statistical Data Warehouse

An SDMX metadata registry plus all of the essential functions needed to load, store, process and publish aggregated statistical data using the SDMX standard.

Java application for on-premise installation on Windows or Linux.

Paid For


Fusion Registry Enterprise Edition

Enterprise-Grade Statistical Data Warehouse

An enterprise-grade solution for collecting, integrating, storing, processing and publishing aggregated statistical data at scale using the SDMX standard.

Annual subscription and perpetual license options.

Java applications for on-premise installation on Windows or Linux.

Fusion Registry Cloud

Fusion Registry Enterprise Edition delivered as a managed service

An enterprise-grade solution for collecting, integrating, storing, processing and publishing aggregated statistical data at scale using the SDMX standard, delivered as a managed service.

Annual subscription.

Compare Features

Community

SDMX 1.0,2.0,2.1 & EDI
SDMX Structural Metadata
Data Validation
Data Structure Mapping
SDMX REST API
Unlimited Structures
Unlimited Users

Data Essentials

SDMX 1.0,2.0,2.1 & EDI
SDMX Structural Metadata
Data Validation
Data Structure Mapping
SDMX REST API
Extended REST API
Basic Data Collection
Data Storage (SQL database only)
Data Dissemination
Fusion Data Browser
Unlimited Structures
Unlimited Users
Data Limits: 100,000 series, 1 million observations
One Data Provider

Cloud

SDMX 1.0,2.0,2.1 & EDI
SDMX Structural Metadata
Data Validation
Data Structure Mapping
SDMX REST API
Extended REST API
Full Data Collection
Data Storage (all options)
Data Dissemination
Fusion Data Browser
Reference Metadata
Data Portal
Data Integration
Content Security
IMF NSDP
IM-JS and SDMX-UI
Unlimited Structures
Unlimited Users
Data Limits: 30 million observations
Unlimited Data Providers

Enterprise

SDMX 1.0,2.0,2.1 & EDI
SDMX Structural Metadata
Data Validation
Data Structure Mapping
SDMX REST API
Extended REST API
Full Data Collection
Data Storage (all options)
Data Dissemination
Fusion Data Browser
Reference Metadata
Data Portal
Data Integration
Content Security
IMF NSDP
IM-JS and SDMX-UI
Load balancing / high availability
Audit
Fusion Edge Server
Unlimited Structures
Unlimited Users
Unlimited Data
Unlimited Data Providers

Function Feature Community Data
Essentials
Cloud Enterprise
Internal Information Model SDMX 2.1
SDMX Input / Output Support SDMX-ML(1.0, 2.0, 2.1), SDMX-JSON(1.0), SDMX-EDI
Structural Metadata Management Author and maintain structural metadata using the web interface
Author and maintain structural metadata using FusionXL addin for Excel
ACID structure repository
Structural metadata import / export as SDMX
Structural metadata import / export as Excel
SDMX structures REST API
Structure storage (no limits)
Structure comparison
Structure validation on import
Structure referential integrity enforcement
Time dependant constraints
Structure-level validity
Item-level validity
Subscription / notification of changes to structures via email or HTTP POST
Structure update transaction journalling
Rollback structure changes to a specified journalled transaction
SDMX Structures Agency Schemes
Attachment Constraints
Categorisations
Category Schemes
Codelists
Concept Schemes
Data Consumer Schemes
Data Provider Schemes
Data Structure Definitions
Dataflows
Hierarchical Codelists
Metadata Structure Definitions
Metadataflows
Organisation Unit Schemes
Processes
Provision Agreements
Reporting Taxonomies
Structure Sets
Fuson Registry 'Extended' Structures Validation Schemes
Reporting Templates
Structural Metadata Storage Options MySQL
Oracle
Microsoft SQL Server
Identity and Access Management Microsoft Active Directory service integration
OpenLDAP directory service integration
Fusion Security directory service integration
Single-user mode (no directory service required)
Certificate autentication
Data Load From File or URL SDMX Data Formats (SDMX-ML, SDMX-JSON, SDMX-CSV, EDI)
Excel
Fusion CSV (non-SDMX variant)
Interactive data load using the web user interface
Interactive store (publish) of loaded data through the web user interface
Programmatic data load and store using the REST API
Plugin interface for adding custom data readers
Data Collection Control how data is collected from providers using SDMX Provision Agreements
Data Providers One Multiple Multiple
Reporting Constraints - control what data each provider can report
Registration - allows data providers to report by registering a file or SDMX REST URL for each dataset
Data ETL Query-for and retrieve data on-demand from SQL databases in the Fusion Registry defined schema (MySQL, Oracle and SQL Server)
Extract, map and load data from arbitrary SQL database schemas Unsupported in the current release
Big Data sources (e.g. HDFS) Unsupported in the current release
Data Portal (beta)
(retrieve and automatically integrate selected data from distributed data services on a defined schedule)
Fusion Registry data sources
SDMX compliant REST API data sources
ECB
BIS (private and public)
IMF
Eurostat
Plugin interface for adding additional data sources
Data Integration - Supported Data Sources Fusion Registry managed local data stores
Distributed Fusion Registries
Data Portal Sources
Data ETL Sources
Data Validation Structural compliance
Compliance of data values with the DSD
Compliance with constraint rules
Duplicate observations or series checks
Mandatory attributes
Validation Schemes - basic business rule validation expressions
Validation Schemes - advanced business rule validation expressions
Validation rules configuration - set what rules are applied and whether they prevent conversion or data publication
Plugin interface for adding custom validators
Data validation processing (load data, validate it against the defined structure and business rules, return a validation report plus valid and invalid series as separate output datasets)
Data validation REST API
VTL VTL 2.0 Unsupported in the current release
SDMX Structure Maps Concept Scheme Maps
Codelist Maps
Data Structure Maps
Dataflow Maps
Regex pattern mapping rules
N-to-N data maps
N-to-1 data maps
1-to-N data maps
1-to-1 data maps
Mapped Datasets Mapped Datasets (like SQL 'views', create new datasets in the statistical data warehouse by applying SDMX Structure Maps to existing datasets)
Data Transformation Data transformation processing (load data, transform it to a different structure using SDMX Structure Mapping and return the result)
Data transformation REST API
Time Series Calculations / Formulae (beta) Calculated Datasets (like SQL 'views', create new datasets in the statistical data warehouse using functions / formulae written in Metadata Technology's expression language)
Data Query Calculations (use functions / formulae written in Metadata Technology's expression language in data queries to perform calculations on the results)
Data Storage SQL Databases (MySQL, Oracle, SQL Server)
Fusion Data Store high-performance in-memory database
Data Conversion Between SDMX formats and versions (SDMX-ML, SDMX-JSON, SDMX-CSV, EDI)
Excel
Fusion CSV (non-SDMX variant)
Data Reporting Excel Reporting Templates - definition and generation
Excel Reporting Templates - validation and load of reported data
Data Dissemination SDMX Data REST API
Availability API
Aggregation queries - aggregate REST API query results on-the-fly
Non-SDMX REST query response formats (Excel, CSV, RDF, Fusion-JSON)
Query response data compression
Embeddable and shareable live charts and tables (create using Fusion Data Browser)
Tableau Data Connector
IMF compliant National Summary Data Page (NSDP)
Fusion Cache (server-side query result caching)
HTTP 304 'Not Modified' support (client-side caching)
Fusion Data Browser
(web data explorer for searching, querying, charting and retrieving data)
Configurable tree view for drilling into the data catalogue by concept
Series list search results with filters
Multi-dimensional query builder
Free-text search
Saved queries
Save workspace
Series basket
Pivot table
Charts (time series, bar, pie, pyramid, heatmap)
Geospatial maps Unsupported in the current release
Export data (Excel, CSV, SDMX)
Create embeddable charts and tables
Fusion Edge Server
(scalable, high-performance distributed data dissemination module)
SDMX Data REST API
Horizontal scaling with load balancer support for clusters of Edge Servers
Embargo - publication of data at a precise time
Selective publication - choose exactly which datasets to publish
Full refresh or incremental data updates
Content Security Structure creation and maintenance controls (Agency privileges)
Data load and modification controls (Data Provider privileges)
Data access controls at the dataflow, series or observation level
Structure access controls at the structure or item level
Enterprise Deployment Audit
Load balancer support (for load-balancing multiple Fusion Registry instances)
'Varnish' web cache / reverse proxy integration
Apache Kafka Integration Kafka 'Structure Notification' Producer for publishing changes to structural metadata on a definable topic
Client-side API IM-JS JavaScript client-side programming library
SDMX-UI TypeScript client-side programming library (higher-level abstraction)
Command Line Interface Fusion Command Line Interface
Fusion Registry 10 Excels at Managing Aggregated Data

Here's how it fits into the official statistics lifecycle.

Putting The Pieces Together

What's required to implement a functional Fusion Registry based SDMX system?

Fusion Registry comprises a suite of modules and components centred on the Fusion Registry Core - the main statistical data engine which incorporates the SDMX Metadata Registry. Here's an example configuration for public data dissemination which illustrates the key modules, components and integrations.
There's a whole lot more. Take a look at the Fusion Registry Ecosystem.

1. SDMX Metadata Registry
The central repository and maintenance tool for all of the SQL structural metadata including Codelists, Concepts and Data Structure Definitions.

2. Fusion Registry Core
Statistical data engine.
The main statistical data engine responsible for all data collection, integration processing and storage. It also includes an optimised query execution engine and SDMX data and metadata REST API allowing it to be used on its own for applications like internal statistical data warehouses.

3. SQL Database Storage
Fusion Registry Core can be configured to use one or more standard SQL databases for metadata and data storage. You'll need to provide your own database but MySQL, SQL Server and Oracle are all supported.

4. Fusion Edge Server
Distributed dissemination server.
Fusion Edge Server is a separate light-weight dissemination module that works in tandem with Fusion Registry. It's designed for public dissemination use cases that need scalability, high-performance and resilience, and where it is desirable to isolate the master Fusion Registry. Fusion Edge Server has no user interface and provides a read-only SDMX REST API suitable for driving a range of dissemination services and tools.

5. Fusion Data Browser
Web tool for data search, visualisation and analysis.
Fusion Data Browser is a web application suitable for both internal and external users to search and explore the catalogue of datasets, select series and cubes of interest, build visualisations using tables and charts, and retrieve selected data in a variety of formats including SDMX, Excel and CSV.

6. NSDP
IMF compliant data dissemination web page.
The National Summary Data Page (NSDP) is a web application that automatically summarises selected datasets on a single web page in a format compliant with the IMF's requirements for countries subscribing to the eGDDS, SDDS and SDDS Plus dissemination standards.

7. IM-JS and SDMX-UI
JavaScript and TypeScript libraries for developing custom data-driven web applications like data portals and dashboards.

8. Excel
Microsoft Excel can be used for (a) preparing and submitting data, (b) authoring and maintaining structures, and (c) analytics of published data.

9. Tableau
Connect Tableau directly to Fusion Registry's data dissemination services for analytics and for building rich visualisations / dashboards.

10. Active Directory or OpenLDAP Service
A Microsoft Active Directory or OpenLDAP directory service is required for authentication and role management.