Bulk datasets information pack
This section is Guidance
Changes post EU Exit and File Descriptions
Changes
From January 2022, changes to how trade in goods statistics are collected necessitate the following changes:
-
- The exports data file (BDSExpYYMM) will include the export of goods from the UK, combining the previous separated non-EU exports and EU dispatches.
- The imports data file (BDSImpYYMM) will include the import of goods to the UK, combining the previous separated non-EU imports and EU arrivals.
Users who require trade data to cover all UK imports and exports, should exploit the interactive trade database on www.uktradeinfo.com or the new API tool.
File Descriptions
The Control File, Exports Data Files and Imports Data Files listed and described below are large, delimited text files created to a standard format. Once downloaded, you will need analytical or database software e.g. SPSS, SAS, Revolution R Open, Microsoft Access or Freeware Open Office, to enable you to process the file.
Control File
The SMKA12 file contains information about each 8 digit commodity code and its corresponding Standard International Trade Classification (SITC) heading. A record indicator identifies which commodity codes need to be included or excluded from General and Special trade compilations and which commodity codes are subject to suppression. The control file also provides a description for each commodity code and the standard abbreviation for net quantity and supplementary units.
Customs Data Files
The BDSExpYYMM file contains details of goods exported from the UK during the month of account, i.e. the month in which the export declarations are received.
The BDSImpYYMM file contains details of goods imported to the UK during the month of account, i.e. the month in which the import declarations are received.
Importer Details
The Importers file lists the name, address and postal code of businesses that have imported goods into the UK from non-EU countries and from 2022 onwards Imports into GB from EU countries in the given month of account, together with their associated commodity code(s). The file is produced from a Unix environment and is a large flat file created to a standard format. Once downloaded, the file can be processed using Microsoft Excel or Freeware Open Office.
- Importers file – produced from January 2016
Exporter Details
The Exporters file lists the name, address and postal code of businesses that have exported goods from the UK to non-EU countries and from 2021 onwards Exports from GB to EU countries in the given month of account, together with their associated commodity code(s). From January 2021, this file was also include details for Exporters who export from GB to EU. The file is produced from a Unix environment and is a large flat file created to a standard format. Once downloaded, the file can be processed using Microsoft Excel or Freeware Open Office.
- Exporters file – produced from January 2016
Trade by preference
The Trade by preference file (BDSPrefYYMM) contains details of goods imported to the UK during the month of account, i.e. the month in which the import declarations are received. The data shown is broken down by commodity code, country of origin of the good, import regime that was used to import the good, import regime the good was eligible to be imported under and statistical regime based on the customs procedure used to import the good. The file includes imports into the UK from non-EU countries and imports into GB from EU countries. The file is a large flat file created in a comma separated file format. For more detail on using csv files please refer to the following guidance.
- Trade by preference file – produced from January 2022
Technical Specifications
Technical specifications detailing the format and content of each file are contained in individual documents available in the Related Content section of this page.
Data dictionary
General and Special Trade
There are two recognised systems for recording trade – the ‘general trade’ system and the ‘special trade’ system *.
HMRC and ONS applied the general trade system (as described in the IMTS) to compile UK trade statistics up to and including the April 2016 month of account. This includes all merchandise crossing the national boundary of the UK, including goods imported into and exported from customs warehouses and free zones. Imported goods are recorded whether or not at the time of importation they are intended for use in the UK or for re-export. Import statistics therefore include UK goods re-imported and goods imported for processing or incorporation with other goods and subsequent re-exportation. Such re-exports are not distinguished from exports of UK produce.
Following a change in legislation on 1st May 2016, affecting the way in which goods are declared to Customs, UK trade statistics switched to the special trade system. As a result, goods imported into customs warehouses and free zones are only recorded once they are removed and enter free circulation or certain customs procedures (e.g. Inward Processing). Re-exports from customs warehouses and free zones are not recorded under the special trade system. The special trade system is used by Eurostat to compile EU trade statistics.
Goods in transit through the UK (even where transhipment is involved) are not included in the OTS.
Due to the switch to special trade reporting from May 2016, 'Port' data (as compiled from the 'Place of Clearance' data on customs declarations) will appear to be missing when searching for data on goods being removed from customs warehousing. This is because the requirement for approved operators to submit supplementary customs declarations for goods imported into customs warehousing is waived and, with it, the 'Port' data element.
There is no requirement to declare 'Port' data for goods being removed from customs warehousing. However, there is likely to be some 'Port' data available (i.e. from businesses submitting full customs declarations for entry into customs warehousing or those who continue to submit customs supplementary declarations voluntarily). As a result, 'Port' data in these circumstances will be incomplete.
* The key difference between general and special trade is the treatment of goods entering Customs warehouses and free zones. Special trade excludes such movements.
Below Threshold Trade Allocations (BTTA)
For businesses whose trade value is above the exemption thresholds, data is collected via the Intrastat survey. For businesses below these thresholds, trade is estimated using the BTTA process, by initially summing the values of arrivals and dispatches supplied on their VAT returns. The BTTA process estimates the total value, net mass and supplementary units for each combination of 8-digit commodity code and partner country for below threshold businesses, and then aggregates these for publication within Chapter (HS2) value totals.
From January 2016 month of account, the interactive data tables and pre- prepared extracts etc. published on uktradeinfo only include BTTA data at Chapter (HS2) level.
Further information regarding BTTA can be found in the Methodology paper for the Overseas Trade Statistics.
Importer and Exporter Details – Disclosure Control
HMRC applies the following automated disclosure control to importers and exporters names and address details:
Where a specific 8-digit commodity code has less than 3 active importers or exporters for a given month, the name and address will only be reported against the first 2 digits of the commodity code (at CN chapter level). If there are still less than 3 active importers or exporters, the data will be reported against pseudo chapter 99.
Where an 8-digit commodity code has been fully suppressed for commercial reasons, no data will be reported against it, including at CN chapter level.
Where an 8-digit commodity code has been partially or fully suppressed in order to protect the national interest, no data will be reported against it, including at CN chapter level.
For periods January 2016, onwards, businesses have an option to opt-out of Importers and Exporters Details datasets so these datasets will not give accurate business counts even when taking into account the active suppression described above.
A request to exclude your business from the exporter / importer information sent to us by the 15th day of the month will mean removal from the information published the following month. For example, if you send in your request by 15 February, we will make sure your information is not published in the publication released the following month (March).
Suppression and Confidentiality
HMRC is committed to making the maximum amount of trade information available, while employing effective disclosure control. Details can be found in the OTS Policy Statement on Suppressions. The suppression system allows for both commodity code and country confidentiality.
Suppressed commodity code data is included in the download files under actual country codes by transforming the declared commodity code into a pseudo code as follows:
Code digits |
Representative code |
1-3 |
990 |
4 & 5 |
HS Chapter (i.e. first 2 digits of actual commodity code) |
6-8 |
SITC Group (3 digit SITC code which corresponds with the actual commodity code) |
Suppressed country information is included in the download files under the actual commodity code declared with a pseudo country code.
Identifying Suppressions in the bulk datasets
There are three levels of suppression indicated in the first column of the tables:
Level |
Description |
1 |
Complete suppression, where no information is published |
2 |
Suppression of countries and ports, where only the overall total value (£ sterling) and quantity (kg) are published |
3 |
Suppression of countries, ports and total trade quantity, where only the overall total value is published |
Examples of each suppression type (PDF, 113KB)
To aggregate data at SITC 3 digits and above
Select record types:
0 and 1 with appropriate commodity codes
3 with appropriate SITC 3 digit code within pseudo commodity code
To aggregate data at HS chapter level
Select record types:
0 and 1 with appropriate commodity codes
3 with appropriate HS chapter within pseudo commodity code (digits 4 and 5).
Note: Pseudo commodity codes resulting from suppressions only appear on the data file and are not held on the control file.
Suppression in Trade by Preference Data
Data is suppressed at CN8 level using passive confidentiality as per the Overseas Trade in Goods Statistics. No other level of aggregation of Commodity Code than the 8-digit Combined Nomenclature is presented in the trade by preference data. So, unlike the Overseas Trade Statistics this product does not include the withheld data at higher levels of aggregation of the Harmonised System (HS).
This means there will be instances where country total, HS2 totals, HS4 totals, and HS6 totals in this analysis will not be the same as in the main bulk data download files and other Overseas Trade Statistics products
Line Type Variable
To aggregate data at SITC 3 digits and above
Select record types:
1 and 2 with appropriate commodity codes
4 with appropriate SITC 3 digit code within pseudo commodity code
To aggregate data at HS chapter level
Select record types:
1 and 2 with appropriate commodity codes
3 with appropriate HS chapter within pseudo commodity code (digits 4 and 5).
SITC aggregations at 4 and 5 digits
Complete aggregations at SITC 4 and 5 digits can only be made if there are no suppressed commodity codes within the SITC level. Partial aggregations may be possible depending on the type of commodity code suppression present. SITC aggregation indicators show the data that should be included to be compatible with OTS publication. Details of aggregation indicator codes are given in Table 3.
Supporting tables
- Table 1 EU Data Files – Records Output For Suppressions (PDF, 217.2 KB)
- Table 2 Non EU Data Files – Records Output For Suppressions (PDF, 219.2 KB)
- Table 1 SITC Aggregation Indicators (PDF, 121.6 KB)
- Table 2 SITC Quantity Conversion Indicators (PDF, 117.2 KB)
- Table 3 Procedure Codes (Export) Definitions (PDF, 125.1 KB)
- Standard Abbreviations Used On The Control File (Quantity Alpha And Commodity Alpha Fields) (PDF, 363 KB)