Full report: https://www.tweedekamer.nl/kamerstukken/detail?id=2024D17779&did=2024D17779

Examining confirmation bias using the F-test

In the short-stay visas application process of the Dutch Ministry of Ministry of Foreign Affairs (known as Kort Verblijf Visum, or KVV), a rule-based classification model to categorise applicants into a fast, regular or intensive track. The goal of this experiment is to examine “whether labeling visa applications has an effect on decisions made by civil servants reviewing the case?”.

Based on the outcomes of a field experiment, in which 42 fictional cases are presented to civil servants, a one-sided F-test (ANOVA, fixed effects, omnibus) is applied. For significance level 5%, no evidence is found that labeling had an effect on the taken decisions.

This form of hypothesis testing is preferred over more advanced form of causal inference, such as inverse probability weighting (IPW) and instrument variable (IV) analysis. See also artikel.