Data Reporting Solutions
for Official Statistics
Reporting Statistical Data using Fusion Registry 9
Metadata Technology’s Data Reporting solutions are designed for Central Banks, International Organisations and National Statistics Offices who need to provide systematic and efficient ways for their reporting agencies to report data.
Reporting Data using Excel Forms
Excel Data Reporting Templates is a new feature for Fusion Registry 9.2 that simplifies manual data reporting processes by supplying data providers with personalised Excel data reporting forms to complete.
The layout of each form is based on a template defined by the data collecting organisation which is created using Fusion Registry's template design tool. The designer can choose from a range of configuration options including setting specific cell colours to identify important attributes such as observation confidentiality. They can also define validation rules.
A unique form is generated for each Dataflow that a Data Provider is required to report data for. If Reporting Constraints have been defined, Excel's cell protection functions are used to restrict the observations that can be entered. This not only simplifies the process for the data provider, but also reduces the risk of data being reported against the wrong codes.
The generated spreadsheets are simple Excel workbooks which use only standard Excel features and formulas. No macros, VBA or addins are used which could violate information security policies.
There's more in the Excel Reporting Templates White Paper.
Note that Fusion Registry Community Edtion can be used to design templates, generate blank spreadsheet forms and validate completed forms, but cannot load and store the reported data.
Submitting Reporting Data through a Web User Interface
Data Reporters can submit prepared reporting data by uploading files using Fusion Registry 9's web user inteface.
A range of file formats are supported including SDMX-ML (XML), CSV and Excel. The CSV and Excel data must be in a format recognised by Fusion Registry - there's more detail about how reporting data should be structured in the Fusion Registry Data Formats Guide.
Reporters also have the option of loading data from a URL which may be useful if it is already published on a national statistics web site.
Only authenticated users with the 'Data Provider' role can report data using the web user interface.
Reporting Data using the REST API
Data providers can use the Fusion Registry 9 Enterprise Edition REST API to report data programatically.
How does it work?
The POST method is used to submit data to Fusion Registry’s REST URL which is of the general format:
Data submissions are handled asynchronously which avoids the risk of client timeouts when reporting large datasets. A unique tracking token is returned as part of the JSON response to the initial POST request which can be subsequently used to monitor progress of the submission by making GET calls to another REST service. Here's an example:
The service will respond with JSON which includes details about the progress of the data submission: PENDING, QUEUED, PROCESSING, PROCESSED, INDEXING, SUCCESS, ERROR.
There's more on how to use the REST API for submitting data in the Web Services API Reference Guide.
Validating and Transforming Data Ready for Reporting
Fusion Registry 9 can be used to provide data reporters with interactive web interface tools for validating reporting data, and transforming it from one format to another in preparation for reporting. There are also REST API methods for data validation and transformation which are suitable for data providers who want to automate their reporting processes.
Fusion Registry’s SDMX data validator conducts a comprehensive range of checks to verify that submitted data complies with the data specifications defined by the data collector in the SDMX Structural Metadata.
Here's what the validator checks for:
- Semantic Compliance - checks that the type of data supplied for each field matches that expected
- Structural Compliance - checks that the structure of the data in terms of its dimensions and attributes matches that defined by the Data Structure Definition
- Representation Compliance - checks that valid codes have been supplied for enumerated dimensions and attributes, and that values comply with the defined representation type for non-enumerated
- Constraint Compliance - checks that the data supplied complies with any defined reporting constraints (e.g. Frequency must be Annual)
- Calculation Compliance - checks that the data complies with any defined Validation Rules (Validation Rules are calculations or check-sums written using Metadata Technology's validation language)
- Duplicate Observations - checks for duplicate observations or series that cannot be merged because they are contradictory (e.g. two different observations for the same time period)
- Mandatory Attributes - checks that compliant values have been supplied for any mandatory attributes
Data Transformation Between Formats
Once successfully validated, data can be converted into a range of alternative formats including SDMX-ML (XML), SDMX-JSON, CSV and Excel.
This supports the use case where reporters prepare data in CSV or Excel, and convert it to SDMX for reporting.
Data Transformation Between SDMX Versions
Data can be freely converted between versions of SDMX - for instance SDMX 2.1 to 2.0. This is particularly useful where the data reporter can supply data in a different version of SDMX to that accepted by the collector.
Data Transformation Between Data Structures
Data can also be transformed into a different structure using SDMX Structure Maps. Structure Maps, which are generally defined by the data collector, allow both the data's demensionality and coding schemes to be changed according to specified rules. There more about Structure Mapping in the Fusion Registry Structure Mapping Guide.