Bulk datasets information pack
This section is Guidance
Changes post EU exit and file descriptions
In January 2022, we made the following changes to our bulk data sets. This was to make them more compatible with post-EU exit collection methodologies.
Exports data file (Bulk Datasets Export YYMM (BDSExpYYMM)):
- now includes the export of goods from the UK
- combines the previously separated non-EU exports and EU dispatches
Imports data file (Bulk Datasets Import YYMM (BDSImpYYMM)):
- now includes the import of goods to the UK
- combines the previously separated non-EU imports and EU arrivals
If you need trade data to cover all UK imports and exports, use our regional or overseas interactive trade tables. Or you can use the API tool. All of these tools are on this site.
File descriptions
The following files are large delimited text files created to a standard format:
- Control file
- Exports data files
- Imports data files
Once you have downloaded these, you will need analytical or database software to process the file.
Control file
This file contains information about:
- each 8-digit commodity code
- its corresponding Standard International Trade Classification (SITC) heading
A record indicator shows:
- which commodity codes need to be included or excluded from general and special trade compilations
- which commodity codes are subject to suppression.
The control file also provides a description for:
- each commodity code
- the standard abbreviation for net quantity and supplementary units
Import and export data files
The BDSExpYYMM file contains monthly details of goods exported from the UK.
The BDSImpYYMM file contains monthly details of goods imported to the UK.
Importer details
The Importers file has been produced since January 2016. It lists the following from businesses that have imported goods into the UK from non-EU countries:
- name
- address
- postal code
- associated commodity codes
Since 2022 it has also listed monthly imports into Great Britain (England, Scotland and Wales - excludes Northern Ireland) from EU countries. This is together with their associated commodity code(s). This part of the file does not include EU to Northern Ireland imports.
The file is created with an open standard format. Once downloaded, use analytical or database software to process it.
Exporter details
The Exporters file has also been produced since January 2016. It lists the same details as the Importers file from businesses that have exported goods from the UK to non-EU countries.
Since 2021, it has also listed monthly exports from Great Britain to EU countries. This is together with their associated commodity code(s).
Since January 2021, this file has also included details for exporters who export from Great Britain to the EU.
The file is also created with an open standard format.
Trade by preference
The Trade by preference file (BDSPrefYYMM) has been produced since January 2022. It contains monthly details of goods imported to the UK.
The data shown is broken down by:
- commodity code
- country of origin of the goods
- import regime used to import the goods
- import regime the goods were imported under
- statistical regime based on the customs procedure used to import the goods
The file includes imports into the UK from non-EU countries and imports into Great Britain from EU countries. It is a large flat file created in a Comma Separated Values (CSV) file format. If you need more information, please refer to the government guidance on using CSV files.
Technical specifications
Find technical specifications under 'Related content' links on this page.
Data dictionary
Find out what each bulk dataset field means in our Data dictionary.
General and special trade
There are two recognised systems for recording trade:
- the ‘general trade’ system
- the ‘special trade’ system
The key difference between them is the treatment of goods entering Customs warehouses and free zones. Special trade excludes such movements.
HM Revenue and Customs and the Office for National Statistics (ONS) applied the general trade system to compile UK trade statistics up to and including April 2016.
This includes all goods in these scenarios:
- crossing the UK's national boundaries
- imported into and exported from customs warehouses and free zones
Imported goods
These are recorded whether they are intended for use in the UK or for re-export at the time of importation.
That means re-exports are not distinguished from exports of UK produce. So import statistics include:
- UK goods re-imported
- goods imported for processing or incorporation with other goods and re-exportation afterwards
UK trade statistics switched to the special trade system, after a change in legislation on 1 May 2016. This affected the way in which goods are declared to Customs.
It means goods imported into customs warehouses and free zones are only recorded once they have:
- been removed
- entered free circulation or certain customs procedures (for example, Inward Processing)
Re-exports from customs warehouses and free zones are not recorded under the special trade system. The latter is used by Eurostat to compile EU trade statistics.
Goods in transit through the UK are not included in the OTS.
Customs warehousing and incomplete port data
Port data is compiled from the 'Place of Clearance' data on customs declarations. But, it does not need submitting on all customs declarations. So some trade data is released by Port as 'not collected'.
Below Threshold Trade Allocations (BTTA)
This applies to trade between Northern Ireland and the EU falling below Intrastat thresholds.
For businesses whose trade value is above the exemption thresholds, data is collected via the Intrastat survey.
For businesses below these thresholds, trade is estimated using the BTTA process. This is done by initially summing up the values of arrivals and dispatches supplied on their VAT returns.
The BTTA process estimates the following for each combination of 8-digit commodity code and partner country for below threshold businesses:
- total value
- net mass
- supplementary units
These are then aggregated for publication within Chapter (Harmonised System 2 (HS2)) value totals.
Since January 2016, the interactive data tables and bulk datasets we publish have only included BTTA data at Chapter (HS2) level.
Further information about BTTA can be found in the Methodology paper for the Overseas Trade Statistics.
Importer and exporter details – disclosure control
HMRC applies automated disclosure control to importers and exporters names and address details if any of these three scenarios listed are met.
Scenario 1
A specific 8-digit commodity code has less than three active importers or exporters for a given month.
In this case, the name and address are only reported against the first two digits of the commodity code. This is at the Combined Nomenclature (CN) chapter level.
If there are still less than three active importers or exporters, the data gets reported against chapter 99. This is a non-trade chapter.
Scenario 2
An 8-digit commodity code has been fully suppressed for commercial reasons. In this case, no data gets reported against it, including at CN chapter level.
Scenario 3
An 8-digit commodity code has been partially or fully suppressed to protect the national interest. In this case, no data gets reported against it, including at CN chapter level.
Since January 2016 businesses have been able to opt out of Importers and Exporters details datasets. This means those datasets will not give accurate business counts. This is even when considering the active suppression described above.
Exclusion requests monthly deadline
The 15th day of the month is the deadline for traders to request exclusions from the exporter / importer information. Details will then be removed the following month.
For example, requests received by 15 February, will result in exclusions from the publication released in March.
Suppression and confidentiality
HMRC is committed to maximising the amount of trade information available, while applying effective disclosure control.
Details can be found in the OTS Policy Statement on Suppressions.The suppression system allows for both commodity code and country confidentiality. This policy also explains the two confidentiality systems - passive and active.
Suppressed commodity code data is included in the download files under actual country codes by turning the declared commodity code into a non-trade code as follows:
|
Code digits |
Representative code |
|
1 to 3 |
990 |
|
4 to 5 |
HS Chapter (for example, first 2 digits of actual commodity code) |
|
6 to 8 |
SITC Group (3 digit SITC code which corresponds with the actual commodity code) |
Suppressed country information is included in the download files under the actual commodity code declared with a non-trade country code.
Identifying suppressions in the bulk datasets
There are three levels of data suppression.
Level 1 - complete suppression
No information gets published
Level 2 - suppression of countries and ports
Only the overall total value in £ sterling and quantity in kg are published.
Level 3 - suppression of countries, ports and total trade quantity
Only the overall total value is published.
Below Level 1, there is no suppression of data. Examples of each suppression type (PDF, 133KB)
To aggregate data at SITC 3 digits and above
Select any of these record types:
- 0 and 1 with appropriate commodity codes
- 3 with appropriate SITC 3 digit code within non-trade commodity code
To aggregate data at HS chapter level
Select any of these record types:
- 0 and 1 with appropriate commodity codes
- 3 with appropriate HS chapter within non-trade commodity code (digits 4 and 5).
Note: Non-trade commodity codes resulting from suppressions only appear on the data file. They are not held on the control file.
Suppression in trade by preference data
Data is suppressed at a certain level in the EU's 8-digit coding system, CN, using the passive confidentiality system. This level is CN8 and is in line with the Overseas Trade in Goods Statistics.
It is the only Commodity Code aggregation level presented in the trade by preference data.
It does not include the withheld data at higher levels of aggregation of the HS. This is unlike the Overseas Trade Statistics.
This means there will be instances where country total, HS2 totals, HS4 totals, and HS6 totals in this analysis will differ from:
- main bulk data download files
- other Overseas Trade Statistics products
Line type variable
To aggregate data at SITC 3 digits and above
Select any of these record types:
- 1 and 2 with appropriate commodity codes
- 4 with appropriate SITC 3 digit code within non-trade commodity code
To aggregate data at HS chapter level
Select any of these record types:
- 1 and 2 with appropriate commodity codes
- 3 with appropriate HS chapter within non-trade commodity code (digits 4 and 5).
SITC aggregations at 4 and 5 digits
Complete aggregations at SITC 4 and 5 digits can only be made if there are no suppressed commodity codes within the SITC level.
Partial aggregations may be possible. This depends on the type of commodity code suppression present.
SITC aggregation indicators show the data that should be included to be compatible with OTS publication. Find more details of aggregation indicator codes under 'Supporting information' on this page.