Skip to main content

Algorithm auditing startups – an emerging but troubled sector

by Joy Green, Mar 18
2 minutes read

The nascent sector of algorithm auditing is appearing, with various start-ups arising in the wake of concerns about the human impacts of AI. This is important, because there is a lack of regulation and standards to govern the deployment of AI, despite increasing awareness of the scope for harm and inappropriate use.

silhouette of person on blue digital wallpaper

For example the predictive AI tool HireVue (marketed as a way to assess applicants at job interviews) has been using facial analysis until very recently to help assess competency scores, despite well-documented risks of racial bias in facial recognition systems. This was despite HireVue being unable to explain how exactly facial analysis scores could be related to competence, given the blackbox nature of machine learning systems.

There is an obvious need for independent bodies to review algorithms of this nature before they are deployed, and various startups are stepping into the space. But without a supportive regulatory framework, it seems that start-ups alone are not enough. HireVue’s algorithm has since been through an independent auditing process, which it cited as an exoneration of its methods. However others in the field (such as the AI governance specialist Alex Engler) have claimed that HireVue mischaracterised the results of the audit and actually used it to engage in ‘ethics-washing’ – pointing out that this is easy to do given the lack of regulatory oversight or defined standards. Cathy O’Neil, owner of the algorithm auditing start-up cited by HireVue, agrees, warning of the risk of corruption in the sector unless there is regulation to support its healthy development.




by Joy Green Spotted 37 signals

Joy is a Principal Futurist at Forum for the Future.

Have you spotted a signal of change?

Register to receive the latest from the Futures Centre.
Sign up

  • 0
  • Share

Join discussion

Related signals

Our use of cookies

We use necessary cookies to make our site work. We'd also like to set optional analytics cookies to help us improve it. We won't set optional cookies unless you enable them. Using this tool will set a cookie on your device to remember your preferences.

For more detailed information about the cookies we use, see our Cookies page.

Necessary cookies

Necessary cookies enable core functionality such as security, network management, and accessibility. You may disable these by changing your browser settings, but this may affect how the website functions.

Analytics cookies

We'd like to set Google Analytics cookies to help us to improve our website by collecting and reporting information on how you use it. The cookies collect information in a way that does not directly identify anyone. For more information on how these cookies work, please see our 'Cookies page'.