Auditing is a multi-faceted concept. In the financial industry, an audit refers to an independent examination of book keeping. According to a pre-defined criteria a financial audit attempts to ensure that books of accounts are properly maintained. For AI systems, auditing is a relatively new concept and auditing criteria have not taken full shape. Due to it’s context dependency, gaining case-based experience in dealing with concrete normative dilemmas in semi-automated decision-making processes is key. Since these dilemmas involve ethical considerations and trade-offs, they cannot be solved by technical audit standards only, but must involve collective deliberation by all stakeholders. Algorithm Audit’s audits are therefore inclusive and deliberative in nature, which resonates with the meaning of the Latin word audire, from which “audit” is derived, meaning “to hear”.

Basically every organization, public or private, big or small, is welcome to present a case. Depending on the nature of the case, the Algoprudence Team decides whether the case is suitable for review by a normative advice commission. You can submit here a case for review.

We are committed to balanced, careful and independent review of ethical issues. We depend on the experts in our normative advice commissions to carry out these evaluations. And we serve the community by making all our advice and knowledge public. Working nonprofit suits our activities and goals best. There are not for-profit legal entities related to Algorithm Audit.

All our cases and corresponding advice are made publicly available, increasing collective knowledge how to devise and use algorithms in an ethical way. In that sense, our work can be considered as open source. However, we only publish the initial problem statement and the final normative advice document in our case repository. Other materials and documentation internal to the case are not shared. All the code developed for this website and for the technical tools we develop, maintain and test are open-source and can be found in Github.

The normative advice commissions that carry out the reviews are independent, academic, multi-disciplinary and diverse. The people we ask to join a commission have an outstanding academic and public track record in the field relevant to the case. Participating experts have no ties to the algorithmic methods or organizations under review. We work together with partners under explicit conditions to avoid ethics washing.

Algorithm Audit is a nonprofit organization supported by independent public funding. Occasionally, case-specific expenses are covered by partners under conditions that guarantee independent evaluation by our normative advice commissions and transparant publication of results. Read more about ways to collaborate together with us.

We are always happy to receive new cases for review. And we are always open to discuss issues and knowledge about pressing ethical issues. Submit here a case for review.

Ethics washing refers to the practice where a company or organization pays lip service to ethics or ethical review, to make it seem outwardly as though it operates in a responsible way. However, the company or organization actually does not wish to implement responsible practices in a meaningful way. We do not participate in ethical reviews as branding. As an organization, we do not confer ethics certificates or labels to our partners. We clearly communicate that our advice commissions only give normative advice and may not be used as stand-in for internal control or external accountability. This is established in explicit conditions under which we work and that we discuss with our partners beforehand. A concrete measure we will regularly adopt is making the partner anonymous in our publication of the case and reports.