Facilitating Legitimate Trade and Stopping Harmful Trade

Considerations on the effectiveness of customs controls and their impact on the environment, health, and other societal goals

Companies and customs agencies working with the German Alliance for Trade Facilitation aim to make trade processes simpler. Both sides of the tradeline can benefit from time and costs saved. Risk management and digital data exchange are the major tools to achieve that. The German Alliance helps introducing them in many projects around the world.

But changing established procedures often makes people feel uncomfortable. Customs officers that were used to certain processes, may wonder “How can I trust the new processes?”. For customs it is not only important to have efficient processes, but also effective controls. Trade should flow as easy as possible, as long as it is safe and legitimate trade.

The question “What do we know about the effectiveness of customs controls?” is quite central for our partners in German Alliance projects. This article serves to deepen the understanding of the issues at stake. It is based on World Customs Organization (WCO) guidelines, academic research and exchange with practitioners. Four take-aways are identified that may inform future project implementation or contribute to the exchange within and beyond the German Alliance for Trade Facilitation.

What is the challenge?

If you hear that port authorities detected 15,000 packages of smuggled cigarettes in 2020 and this being an increase of 25% compared to 2019, you may think customs controls have become more effective (fictitious example).

You could be misled. Maybe criminals tried to smuggle much more in 2020 than in 2019, and port authorities detected a lower percentage in 2020. Then the number of detected packages number increases, but more cigarettes in absolute and percentage terms pass through.

A diagram presented at the WCO Picard Conference 2020 illustrates the challenge.

Source: Own illustration according to Mathieu Labare & Jonathan Migeotte, Belgian Customs, presentation held at Picard Conference, Nov. 24, 2020

Customs usually selects a subset of customs declarations for inspection. Some of the selected declarations will turn out to be irregular and become detected declarations.

Customs can observe the total number of declarations, the number of selected and the number of detected declarations (green, yellow and blue squares). However, customs cannot observe the number of irregular declarations (red square).

The red area may in reality be larger, and the yellow square may be placed lower. The blue square would remain of the same size, but the overall effectiveness of control would be much worse.

What does the World Customs Organization say?

The WCO Risk Management Compendium presents a maturity model with five different stages of risk management, ranging from an ad hoc approach to an optimized and integrated approach. On the topic of effectiveness of customs controls it points out that customs could only know the effectiveness of their controls if they applied statistically valid random sampling.

In June 2019 the WCO established a Working Group that is developing a Performance Measurement Mechanism (PMM). Effectiveness of customs controls is one among other subjects. Related outcomes listed in the PMM are, among others:

  • Increased effectiveness in safeguarding public health
  • Increased effectiveness in combatting drugs trafficking
  • Increase effectiveness in the fight against environmental threats

Despite the WCO emphasis on statistical random sampling, early discussions (Nov. 2021) indicate that the measurement of these outcomes may rely very much on seizures, i.e. detection numbers. Just like the example at the start of this article.

One reason is confidentiality. Customs agencies believe that sharing information on selection techniques may help fraudsters “to game the system” and thus avoid detection. For this reason, it is very difficult to discuss sampling techniques in a broader group. It is also the reason why there is very little academic literature on the topic. Two exceptions are presented in the next section.

What do we know from academic literature?

One of the few publications on the effectiveness of customs control is Performance measurement of the KCS customs selectivity system (Han and Ireland, 2014). The paper analyses three different selection mechanisms applied by Korea Customs Service (KCS): Manual selection (MN), Rule-based selection (RB), and Random selection (RD). They compare the detection rates of the three, and how these change if the selection rate is increased[1].

The findings are not easy to interpret:

  • Finding: Decrease of detection rate for MN in reaction to increase of selection rate.
    Possible interpretation: It seems that KCS uses MN quite well. They still find irregular declarations, but at a lower rate.  
  • Finding: Increase of detection rate for RD in reaction to increase of selection rate.
    Possible interpretation: If selection is really random, the detection rate should not change irrespective of the selection rate…
  • Finding: Non-significant change of detection rate for RB
    Possible interpretation: KCS could increase the selection without a loss of detection efficiency (so probably should do it?).

In another paper published by Kim et al in 2020 the researchers developed a self-learning algorithm, called DATE, for detecting illicit trade transactions. [2] They compare the performance of DATE with other rule-based algorithms. Three insights from the paper stand out:

  • The performance indicators they use are: “precision” (relation of: illicit transactions / inspected transactions), “recall” (detected transactions / all illicit transaction), and “revenue” (revenue through inspection / would-be revenue of all illicit transactions).[3]
  • The rule-based algorithms reduce the performance gap compared to DATE when the amount of training data is increased.
  • “Interpretability” is an important feature for any system, i.e. customs officers are able to understand why a system or algorithm is successful (and not “a black box”).
“Customs control effectiveness” – what does it mean in practice?

In April 2022, the author had an exchange with two customs project partners on the topic of control effectiveness. The aim was to get a better idea what customs authorities have to say on the topic, and how they measure it.

In both cases, it turned out that “effectiveness” was not interpreted as effective selection what to control, but how to conduct an effective control of the shipments that are selected. This snapshot is only anecdotal evidence. But it may indicate that customs authorities in developing countries don’t reflect much on what actual detection tells them about their selection technique. Probably there are other more pressing priorities of immediate day-to-day relevance. Nevertheless, some practical considerations can be made that seem to be relevant for change processes at customs.

Practical considerations and take-aways

Not all customs authorities are at a risk management maturity stage to introduce statistically valid random sampling techniques soon. However, a few take-aways could be useful also at earlier stages of upgrading risk management systems.

  1. Promote a solid understanding of the statistical context and of the difference between observed and unobserved cases. This is a precondition to develop a sound risk management approach. It may also help to prevent that customs focuses too much on isolated Key Performance Indicators (KPIs) (high detection rates as such).
  2. Start to develop indicators for the effectiveness of customs controls. Even if customs agencies cannot yet deploy sophisticated statistical methods, it will be useful to reflect on customs control practices, on objectives and to identify respective indicators. Equally important is to reflect on the interpretation of the indicators.
  3. Use the insights from the KCS study to observe how an increased selection rate influences the effectiveness of different selection mechanisms. This does not require sophisticated random sampling. It could yield interesting insights that could be used for awareness building among customs officers and ultimately increase ownership and motivation for an improved risk management.

Finally, the author would encourage all customs authorities to take part in WCO’s work on performance measurement and defining meaningful KPIs. Our experience in the German Alliance is that collaboration is a key to unlock the positive potential of trade. The secretariat of the German Alliance looks forward to further exchange and applying insights to the projects.

Author: Karl Bartels, Project Manager, Monitoring & Evaluation Specialist, German Alliance for Trade Facilitation

Photo credits: bluedesign – stock.adobe.com

[1] This is done based on knowing that the selection rate was increased and the change of the ratio (detection rate / selection rate). The specific detection and selection rates are unknown.
[2] DATE: Dual-Attentive Tree-aware Embedding for Customs Fraud Detection. Kim, Tsai, Singh, Choi, Ibok, Li, Cha. 2020.
[3] Measuring “recall” and “revenue” is only possible with full knowledge of the sample of trade transactions (i.e. 100% of inspections). The paper uses data from Nigeria where this is practised.

More Topics

Ghana: Enabling Women in Cross-Border Trade
The event series ‘Women in Cross-Border Trade’ was initiated in recognition of the need for gender-sensitive approaches in trade facilitation. Relevant …
Facilitating Legitimate Trade and Stopping Harmful Trade
Considerations on the effectiveness of customs controls and their impact on the environment, health, and other societal goals
“Public-Private Partnerships […] – a unique Tool in the Trade Arena“
In an interview with Ms Wing Xiaoying Huo, who works as Senior Director in the GoTrade/Trade Facilitation Department of DPDHL, we talked among others ab…