Bulk datasets information pack

This section is Guidance

Changes post EU Exit and File Descriptions

Changes

From January 2021, changes to how trade in goods statistics are collected necessitate the following changes:

  • The non-EU exports data file (SMKE19) will include the export of goods from GB to the EU.
  • The EU dispatches data file (SMKX46) and the associated EU dispatches non-response file (SESX16) will no longer be published, due to confidentiality concerns arising from changes to the collection of GB to EU export movements.
  • The EU arrivals data file (SMKM46) will no longer include data in the “No. of Consignments” field, as its completion is optional and the data is of insufficient quality.

Users who require trade data to cover all UK imports and exports, should exploit the interactive trade database on www.uktradeinfo.com or the new API tool. 

File Descriptions

The Control File, Non-EU Data Files, EU Data Files and EU Estimate Files listed and described below are large, delimited text files created to a standard format. Once downloaded, you will need analytical or database software e.g. SPSS, SAS, Revolution R Open, Microsoft Access or Freeware Open Office, to enable you to process the file.

Control File

The SMKA12 file contains information about each 9 digit commodity code and its corresponding Standard International Trade Classification (SITC) heading. A record indicator identifies which commodity codes need to be included or excluded from General and Special trade compilations and which commodity codes are subject to suppression. The control file also provides a description for each commodity code and the standard abbreviation for net quantity and supplementary units.

Customs Data Files

The SMKE19 file contains details of goods exported from the UK to non-EU countries during the month of account, i.e. the month in which the export declarations are received. Amendments to entry declarations submitted in a previous month will be shown as amendment records. From January 2021, this file will contain details for Exporters who export from GB to EU as well.

The SMKI19 file contains details of goods imported to the UK from non-EU countries during the month of account, i.e. the month in which the import declarations are received. Amendments to entry declarations submitted in a previous month will be shown as amendment records.

Intrastat Data Files

The SMKX46 file contains details of goods exported from the UK to EU countries, referred to as dispatches, during the month of account, i.e. the reference period to which the goods relate. Late declarations, i.e. transactions which have taken place in an earlier month and revisions to reported trade will be shown as amendment records. This is only available for periods up to and including December 2020.

The SMKM46 file contains details of goods imported to the UK from EU countries, referred to as arrivals, during the month of account, i.e. the reference period to which the goods relate. Late declarations, i.e. transactions which have taken place in an earlier month and revisions to reported trade will be shown as amendment records.

Intrastat Estimate Files

The SESX16 file contains details of the current month non-response estimates for UK exports to EU countries and re-calculated estimates for the previous five months at 2 digit SITC level. This is only available for periods up to and including 2020.

The SESM16 file contains details of the current month non-response estimates for UK imports from EU countries and re-calculated estimates for the previous five months at 2 digit SITC level.

Importer Details

The SIAI11 Importer Details file lists the name, address and postal code of businesses that have imported goods from non-EU countries in the given month of account, together with their associated commodity code(s). The file is a large, delimited text file created to a standard format. Once downloaded, you will need analytical or database software e.g. SPSS, SAS, Revolution R Open, Microsoft Access or Freeware Open Office, to enable you to process the file.

  • SIAI11 – produced until December 2015 - replaced by the Importers file.

The Importers file lists the name, address and postal code of businesses that have imported goods from non-EU countries in the given month of account, together with their associated commodity code(s). The file is produced from a Unix environment and is a large flat file created to a standard format. Once downloaded, the file can be processed using Microsoft Excel or Freeware Open Office.

  • Importers file – produced from January 2015

Exporter Details

The Exporters file lists the name, address and postal code of businesses that have exported goods to non-EU countries in the given month of account, together with their associated commodity code(s). From January 2021, this file was also include details for Exporters who export from GB to EU. The file is produced from a Unix environment and is a large flat file created to a standard format. Once downloaded, the file can be processed using Microsoft Excel or Freeware Open Office.

  • Exporters file – produced from January 2016

Technical Specifications

Technical specifications detailing the format and content of each file are contained in individual documents available in the Related Content section of this page.

General and Special Trade

There are two recognised systems for recording trade – the ‘general trade’ system and the ‘special trade’ system *.

HMRC and ONS applied the general trade system (as described in the IMTS) to compile UK trade statistics up to and including the April 2016 month of account. This includes all merchandise crossing the national boundary of the UK, including goods imported into and exported from customs warehouses and free zones. Imported goods are recorded whether or not at the time of importation they are intended for use in the UK or for re-export. Import statistics therefore include UK goods re-imported and goods imported for processing or incorporation with other goods and subsequent re-exportation. Such re-exports are not distinguished from exports of UK produce.

Following a change in legislation on 1st May 2016, affecting the way in which goods are declared to Customs, UK trade statistics switched to the special trade system. As a result, goods imported into customs warehouses and free zones are only recorded once they are removed and enter free circulation or certain customs procedures (e.g. Inward Processing). Re-exports from customs warehouses and free zones are not recorded under the special trade system. The special trade system is used by Eurostat to compile EU trade statistics.

Goods in transit through the UK (even where transhipment is involved) are not included in the OTS.

Due to the switch to special trade reporting from May 2016, 'Port' data (as compiled from the 'Place of Clearance' data on customs declarations) will appear to be missing when searching for data on goods being removed from customs warehousing. This is because the requirement for approved operators to submit supplementary customs declarations for goods imported into customs warehousing is waived and, with it, the 'Port' data element.

There is no requirement to declare 'Port' data for goods being removed from customs warehousing. However, there is likely to be some 'Port' data available (i.e. from businesses submitting full customs declarations for entry into customs warehousing or those who continue to submit customs supplementary declarations voluntarily). As a result, 'Port' data in these circumstances will be incomplete.


 

* The key difference between general and special trade is the treatment of goods entering Customs warehouses and free zones. Special trade excludes such movements.

Data compilation

For General trade (for UK OTS National Statistics prior to 1st May 2016) use the following:

Exports

include Trade Indicator

0 or 1

 

exclude Suite Indicator

3, 8 or 9

Imports

include Trade Indicator

0 or 4

 

exclude Suite Indicator

3, 8 or 9

Dispatches

include Trade Indicator

0 or 1

 

exclude Suite Indicator

8 or 9

Arrivals

include Trade Indicator

0 or 4

 

exclude Suite Indicator

8 or 9

For Special trade (for Eurostat statistics and UK OTS National Statistics with effect from 1st May 2016) use the following:

Exports

include Trade Indicator

0

 

exclude Suite Indicator

3 or 9

Imports

include Trade Indicator

0 or 5

 

exclude Suite Indicator

3 or 9

Dispatches

include Trade Indicator

0

 

exclude Suite Indicator

9

Arrivals

include Trade Indicator

0 or 5

 

exclude Suite Indicator

9

Below Threshold Trade Allocations (BTTA)

For data that is collected via the Intrastat survey from all businesses above the exemption thresholds. For businesses below these thresholds, trade is estimated using the BTTA process, by initially summing the values of arrivals and dispatches supplied on their VAT returns. The BTTA process estimates the total value, net mass and supplementary units for each combination of 8-digit commodity code and partner country for below threshold businesses, and then aggregates these for publication within Chapter (HS2) value totals.

From January 2015 month of account, the interactive data tables and pre- prepared extracts etc. published on uktradeinfo only include BTTA data at Chapter (HS2) level. The monthly delimited text files supplied in the Data Downloads area continue to include allocations at 8-digit commodity code level (indicated by zeros (000) in the Nature of Transaction Code field), to allow aggregations at Chapter level to match the totals published elsewhere.

Further information regarding BTTA can be found in the Methodology paper for the Overseas Trade Statistics.

Importer and Exporter Details – Disclosure Control

HMRC applies the following automated disclosure control to importers and exporters names and address details:

Where a specific 8-digit commodity code has less than 3 active importers or exporters for a given month, the name and address will only be reported against the first 2 digits of the commodity code (at CN chapter level). If there are still less than 3 active importers or exporters, the data will be reported against pseudo chapter 99.

Where an 8-digit commodity code has been fully suppressed for commercial reasons, no data will be reported against it, including at CN chapter level.

Where an 8-digit commodity code has been partially or fully suppressed in order to protect the national interest, no data will be reported against it, including at CN chapter level.

For periods January 2016, onwards, businesses have an option to opt-out of Importers and Exporters Details datasets so these datasets will not give accurate business counts even when taking into account the active suppression described above.

Suppression and Confidentiality

HMRC is committed to making the maximum amount of trade information available, while employing effective disclosure control. Details can be found in the OTS Policy Statement on Suppressions. The suppression system allows for both commodity code and country confidentiality.

Suppressed commodity code data is included in the download files under actual country codes by transforming the declared commodity code into a pseudo code as follows:

Code digits

Representative code

1-3

990

4&5

HS Chapter (i.e. first 2 digits of actual commodity code)

6-8

SITC Group (3 digit SITC code which corresponds with the actual commodity code)

Suppressed country information is included in the download files under the actual commodity code declared with a pseudo country code.

Identifying Suppressions in the bulk datasets

 There are three levels of suppression indicated in the first column of the tables:

Level

Description

1

Complete suppression, where no information is published

2

Suppression of countries and ports, where only the overall total value (£ sterling) and quantity (kg) are published

3

Suppression of countries, ports and total trade quantity, where only the overall total value is published

Examples of each suppression type (PDF, 113KB)

To aggregate data at SITC 3 digits and above

Select record types:

0 and 1 with appropriate commodity codes

3 with appropriate SITC 3 digit code within pseudo commodity code

To aggregate data at HS chapter level

Select record types:

0 and 1 with appropriate commodity codes

3 with appropriate HS chapter within pseudo commodity code (digits 4 and 5).

Note: Pseudo commodity codes resulting from suppressions only appear on the data file and are not held on the control file.

SITC aggregations at 4 and 5 digits

Complete aggregations at SITC 4 and 5 digits can only be made if there are no suppressed commodity codes within the SITC level. Partial aggregations may be possible depending on the type of commodity code suppression present. SITC aggregation indicators show the data that should be included to be compatible with OTS publication. Details of aggregation indicator codes are given in Table 3.