Risk Profiling for Social Welfare Reexamination (AA:2023:02)

Key takeaways normative advice commission
> Algorithmic profiling is possible under strict conditions

The use of algorithmic profiling to re-examine whether social welfare benefits have been duly granted, is acceptable if applied responsibly.

> Profiling must not equate suspicion

Re-examination needs to be based more on service and less on distrust.

> Diversity in selection methods

To avoid tunnel vision and negative feedback loops, algorithmic profiling ought to be combined with expert-driven profiling and random sampling.

> Well-considered use of profiling criteria

Caring to avoid (proxy) discrimination and other undesirable forms of differentiation, the normative advice commission assessed variables individually on their eligibility for profiling (see Infographic).

> Explainability requirements for machine learning

It is necessary that the sampling of residents can be explained throughout the entire decision-making process. Complex training methods for variable selection, such as the xgboost algorithm discussed in this case study, are considered too complex to meet explainability requirements.

Summary advice

The commission judges that algorithmic risk profiling can be used under strict conditions for sampling residents receiving social welfare for re-examination. The aim of re-examination is a leading factor in judging profiling criteria. If re-examination were based less on distrust and adopts a more service-oriented approach, then the advice commission judges a broader use of profiling variables permissible to enable more precise targeting of individuals in need of assistance. For various variables used by the Municipality of Rotterdam during the period 2017-2021, the commission gives an argued judgement why these variables are or are not eligible as a profiling selection criterion (see Infographic). A combined use of several sampling methods (including expert-driven profiling and random sampling) is recommended to avoid tunnel vision and negative feedback loops. The commission advises stricter conditions for the selection of variables for use by algorithms than for selection by domain experts. The commission states that algorithms used to sample citizens for re-examination must be explainable. Complex training methods, such as the xgboost model used by the Municipality of Rotterdam, do not meet this explainability criterion. This advice is directed towards all Dutch and European municipalities that use or consider using profiling methods in the context of social services.

Presentation
The advice report (AA:2023:02:A) has been presented to the Dutch Minister of Digitalization on November 29, 2023. A press release can be found here.

Algoprudence

Download the full advice report (AA:2023:02:A) here and problem statement (AA:2023:02:P) here.

Normative advice commission

  • Abderrahman El Aazani, Researcher at the Ombudsman Rotterdam-Rijnmond
  • Francien Dechesne, Associate Professor Law and Digital Technologies, Leiden University
  • Maarten van Asten, Alderman Finance, Digitalization, Sports and Events Municipality of Tilburg
  • Munish Ramlal, Ombudsman Metropole region Amsterdam
  • Oskar Gstrein, Assistant Professor Governance and Innovation, University of Groningen

Funded by