Collaborating with

Building AI auditing capacity


from a not-for-profit perspective


Building
AI auditing capacity


from a
not-for-profit perspective


Distinctive in

Independence

By working nonprofit and under explicit terms and conditions, we ensure the independence and quality of our audits and normative advice

Normative advice

Mindful of societal impact our commissions provide normative advice on ethical issues that arise in algorithmic use cases

Public knowledge

Audits and corresponding advice (algoprudence) are made publicly available, increasing collective knowledge how to deploy and use algorithms in an responsible way

Independence

By working nonprofit and under explicit terms and conditions, we ensure the independence and quality of our audits and normative advice

Normative advice

Mindful of societal impact our commissions provide normative advice on ethical issues that arise in algorithmic use cases

Public knowledge

Audits and corresponding advice (algoprudence) are made publicly available, increasing collective knowledge how to deploy and use algorithms in an responsible way

AI expertise

Algorithms for decision support

Auditing data-analysis methods and algorithms used for decision support. Among others by checking organizational checks and balances, and assessing the quantitative dimension

AI Act standards

As Algorithm Audit is part of Dutch and Europen standardization organisations NEN and CEN-CENELEC, AI systems are audited according to the latest standards. See also our public knowledge base on standardization

Profiling

Auditing rule-based and ML-driven profiling, e.g., differentiation policies, selection criteria, Z-testing, model validation and organizational aspects

FP-FN balancing

Context-dependent review of ML and DL confusion matrix-based evaluation metrics, such as False Positives (FPs) and False Negatives (FNs)

Ranking

Recommender systems are everywhere. With the new Digital Services Act (DSA) that came into force last summer, auditing ranking systems is highly relevant

Generative AI

Auditing training process of foundation models, among others selection of training data, human feedback for reinforcement learning and risk management, according to AI Act standards

Algorithms for decision support

Auditing data-analysis methods and algorithms used for decision support. Among others by checking organizational checks and balances, and assessing the quantitative dimension

AI Act standards

As Algorithm Audit is part of Dutch and Europen standardization organisations NEN and CEN-CENELEC, AI systems are audited according to the latest standards. See also our public knowledge base on standardization

Profiling

Auditing rule-based and ML-driven profiling, e.g., differentiation policies, selection criteria, Z-testing, model validation and organizational aspects

FP-FN balancing

Context-dependent review of ML and DL confusion matrix-based evaluation metrics, such as False Positives (FPs) and False Negatives (FNs)

Ranking

Recommender systems are everywhere. With the new Digital Services Act (DSA) that came into force last summer, auditing ranking systems is highly relevant

Generative AI

Auditing training process of foundation models, among others selection of training data, human feedback for reinforcement learning and risk management, according to AI Act standards

Recent audits

Risk Profiling Social Welfare Re-examination

Normative advice commission provides rationales why these variables are eligible or not as a profiling selection criterion for a xgboost algorithm

Technical audit indirect discrimination

Assessment of risk distributions through Z-tests and bias test for various steps in algorithmic-driven decision-making process

Risk Profiling Social Welfare Re-examination

Normative advice commission provides rationales why these variables are eligible or not as a profiling selection criterion for a xgboost algorithm

Technical audit indirect discrimination

Assessment of risk distributions through Z-tests and bias test for various steps in algorithmic-driven decision-making process

Building algoprudence

Step 1

Identifying issue

Identifying a concrete ethical issue in a real algorithm or data-analysis tool

Step 2

Problem statement

Describe ethical issue, legal aspects and hear stakeholders and affected groups

Step 3

Advice commission

Deliberative conversation on ethical issue by diverse and inclusive advice commission

Step 4

Public advice

Advice of commission is published together with problem statement on our website. Publicly sharing the problem statement and normative advice is called algoprudence

Step 1 – Identifying issue

Identifying a concrete ethical issue in a real algorithm or data-analysis tool

Step 2 – Problem statement

Describe ethical issue, legal aspects and hear stakeholders and affected groups

Step 3 – Advice commission

Deliberative conversation on ethical issue by diverse and inclusive advice commission

Step 4 – Public advice

Advice of commission is published together with problem statement on our website. Publicly sharing the problem statement and normative advice is called algoprudence

Advantages of algoprudence

Learn & harmonize

> Ignite collective learning process to deploy and audit responsible AI

> Harmonizes the resolution of ethical questions and the interpretation of open legal norms

Question & criticize

> Fostering criticism on normative decision-making through transparency

> Informing public debate with important ethical issues to be discussed within democratic sight

Inclusion & participation

> Connecting various stakeholders to design ethical algorithms together with technical experts

> European answer to deploy responsible AI systems

Learn & harmonize

> Ignite collective learning process to deploy and audit responsible AI

> Harmonizes the resolution of ethical questions and the interpretation of open legal norms

Question & criticize

> Fostering criticism on normative decision-making through transparency

> Informing public debate with important ethical issues to be discussed within democratic sight

Inclusion & participation

> Connecting various stakeholders to design ethical algorithms together with technical experts

> European answer to deploy responsible AI systems

Jurisprudence for algorithms


The Movie

Newsletter

Stay up to date about our work by signing up for our newsletter

Newsletter

Stay up to date about our work by signing up for our newsletter

Building public knowledge for ethical algorithms