Author: David Swiderski, PTCMW Blog Editor
There has been an increase recently in the discussion of the use of artificial intelligence (AI) in the personnel selection space. One indication of the impact of this area on the field of I/O psychology is the recent announcement of the upcoming 2023 SIOP Leading Edge Consortium (LEC) on Talent Assessment Strategies of the Future with a focus on “new applications of AI to assessment development and scoring”. The intent of this blog is to provide a brief roundup of recently published practical guidance on the use of AI in personnel selection decisions.
AI has not been a focus of traditional education and training in industrial and organizational psychology but is an emerging force affecting how we understand and predict people’s performance at work. It is imperative that those working in this space understand the implications of using these techniques and how they can be applied appropriately. This is especially true in high-stakes contexts that can have a significant impact on individuals’ careers.
The following documents can serve as a starting point for those considering the development, validation, implementation, and maintenance of AI-based assessments.
The Institute for Workplace Equality. (2022). Technical Advisory Committee report: EEO and DEI&A considerations in the use of artificial intelligence in employment decision making. Link
The Institute for Workplace Equality is a non-profit organization that strives to educate businesses on diversity and inclusion practices and equal employment opportunity compliance. The Institute commissioned a 40-member Technical Advisory Committee (TAC) from domains such as industrial and organizational psychology, psychometrics, data science, economics, and employment law. The TAC’s in-depth report approaches the topic of AI in employment decision making from statistical, ethical, and legal perspectives by aggregating information across the diverse group of contributors. Additionally, each section of the report was shaped by the results of a survey of TAC members. Some of the topics covered in the report include:
- Where AI is most prevalent in employment decision making.
- Privacy and fairness issues around how AI data is collected, used, and stored.
- How the concepts and principles described in the Uniform Guidelines on Employee Selection Procedures apply to AI-based assessments.
- Analyzing adverse impact in the context of the unique challenges presented by using AI-based assessments.
Society for Industrial and Organizational Psychology. (2023). Considerations and recommendations for the validation and use of AI-based assessments for employee selection. Link
Undoubtedly, most readers of this blog will be familiar with the Society for Industrial and Organizational Psychology, the premier professional association for I/O psychologists. In addition to the LEC mentioned above, SIOP has acknowledged the growing interest in AI by releasing a statement in January 2022 on the use of AI in hiring and offering a webcast on the topic featuring SIOP members with expertise in this area. The most comprehensive response to date from SIOP in this arena has come in the form of a document from SIOP’s Task Force on AI-Based Assessments. The document details recommendations from the Task Force on developing, validating, and using AI-based assessments. The authors discuss how different elements of AI-based assessments relate to the concepts and procedures discussed in SIOP’s Principles for the Validation and Use of Personnel Selection Procedures. The document delves into issues such as considerations for collecting validation evidence and how the collection of training data in AI-based assessments can influence their scoring and outcomes. The document concludes with a discussion of the information that should be documented as part of the development and validation of AI-based assessments.
Landers, R. N., & Behrend, T. S. (2022). Auditing the AI auditors: A framework for evaluating fairness and bias in high stakes AI predictive models. American Psychologist, 78(1), 36–49. https://doi.org/10.1037/amp0000972
Tara Behrend and Richard Landers have been prominent academic voices on the intersection between technology and human behavior in the workplace for over a decade. Their recent article in American Psychologist focuses on the ethical use of AI through a set of guidelines for conducting audits of AI systems. The authors broadly define high-stakes decisions beyond hiring individuals in organizations to many decisions made across different domains of psychology. The example presented throughout the article focuses on a situation in which an AI algorithm is used to score interview responses. The article highlights 12 components of AI systems that should be included in an audit and identifies important considerations to think through in the evaluation of each component. Finally, the article concludes with an emphasis on the importance of collaboration and having an interdisciplinary lens for the adoption of sound AI auditing practices in the future.
Tippins, N. T., Oswald, F. L., & McPhail, S. M. (2021). Scientific, legal, and ethical concerns about AI-based personnel selection tools: a call to action. Personnel Assessment and Decisions, 7(2), 1-22. https://doi.org/10.25035/pad.2021.02.001
The final document in this roundup is a journal article that outlines 11 concerns with using AI from three prominent figures in the personnel selection field. The concerns touch on many different points in the assessment development and validation process. The article also contains lists of questions that researchers and practitioners should take into account in light of the concerns stated in the article. Finally, the article ends with a call to action encouraging I/O psychologists to play a central role in this space by considering how our professional standards and principles apply to AI-based assessments. Similar to Landers and Behrend, the authors call for using an interdisciplinary approach to make progress in this area.
Note that this list is a brief roundup of several recent committee and refereed journal-produced documents. We encourage interested readers to be on the lookout for additional professional and scientific developments related to this space. If you would be interested in contributing your perspective to a future blog on AI in the workplace, please reach out to blog@ptcmw.org to discuss your ideas.
