We evaluate Large Language Models (LLMs) and other generative AI applications relating to guardrails, privacy and AI Act compliance. Based on a mature RAG application of the Dutch judiciary, we have developed a validation framework to analyze content filters, embedding strategies and user interaction design choices. Read more about AI Safety project work we conduct for the AI Office of the European Commission.

Our open-source AI and Algorithms Qualification Toolkit (AI AQT) helps organizations identify algorithms and AI systems at scale and helps in assigning the appropriate risk category. As a member of Dutch and European standardization organisations NEN and CEN-CENELEC, Algorithm Audit monitors and contributes to the development of standards for the AI Act. See also our public knowledge base on standardization.

We evaluate algorithmic systems both from a qualitative and quantitative dimension, including analysis of objective justification as a key element of EU non-discrimination law. In addition to expertise in data analysis and statistics, Algorithm Audit has legal expertise relating to the GDPR, specifically prohibited automated decision-making, and organizational risk management. See our public standards for the responsible use of algorithmic systems.

We audit algorithmic systems from organisational, technial and legal perspective. We also offer support with interpretation and implementation of the AI Act and GDPR legal texts, annexes and guidelines from the European Commission, including issues regarding definitions, high-risk applications and conformity assessment. Our audit reports and white papers contribute to public knowledge how legal compliance can be realised.