Author: David Swiderski, PTCMW Blog Editor
The 2023 PTCMW Fall Event was headlined by Ann Marie Ryan, a leading scholar in the field of personnel selection. This blog is intended to provide some quick takeaways to spread Dr. Ryan’s message on the influence of technology on accessibility and hiring. Dr. Ryan’s talk focused on three dichotomies that exist when considering how technology has changed the hiring landscape for applicants and hiring managers with implications for researchers who study these concepts and those practicing in this area.
Takeaway 1: Technology can expand and decrease access for individuals
On the positive side, this takeaway focused on the ability of technology to bring assessments to individuals who would have never had the opportunity to otherwise apply for and be evaluated for a job they are interested in. The digital divide is a well-known phenomenon that focuses on differences in access to technology and the internet across different groups. For example, a survey by the Pew Research Center found that 80% of White adults reported owning a desktop or laptop computer while only 69% of Black adults and 67% of Hispanic adults reported owning either of those technologies. With the rise in mobile assessments and unproctored environments, assessments can be taken by a wider group of individuals than ever before.
Dr. Ryan cautioned that as we continue to adopt new technologies, to be aware of the “hidden barriers” that we may be creating for some groups. As an example of a hidden barrier, the speaker cited that some companies are using artificial intelligence to transcribe what individuals are saying in an asynchronous interview and using that to score them on competencies via an algorithm. Although this may be beneficial for some groups, one group that is potentially negatively impacted by this is those who stutter. Dr. Ryan presented some of her work with HeardAI, which aims to improve voice recognition technology for those who stutter. By improving the way that this technology works, this will not only be beneficial for the estimated 80 million individuals who stutter, but also has positive benefits for the 180 million individuals with speech differences, and for any individual who experiences speech disfluency, which happens to everyone. This is an example of what is known as the curb cut effect, where improving the environment for those with a disability also improves functionality for all people. One key to this that Dr. Ryan expressed was that we need datasets that represent all relevant users to the extent possible when training algorithms.
Takeaway 2: Technology can expand what we assess, but can also introduce opportunities for bias
The second dichotomy that the talk focused on was that technology has provided assessment professionals with new ways to assess constructs that are relevant to many different jobs. Cutting edge research on the role of artificial intelligence in selection has shown the ability for artificial intelligence to assist in item development, provide better strategies for combining predictors, and reduce subgroup differences. In providing novel methods for measuring constructs, there may be additional construct-relevance variance that can be captured using these strategies.
On the other side, during the talk Dr. Ryan discussed the potential for these types of assessments to unintentionally introduce opportunities for bias in the assessment process. For example, identity contingency cues may signal to an applicant how welcome they are within an organization by demonstrating how different aspects of their identity are represented and valued by the organization (or not). More complex assessment environments offered by technology may make organizations more or less attractive to applicants depending on their reactions to the assessment and if they see themselves represented in those environments. Having more cues available for applicants to interpret may not always lead to positive outcomes. In general, it appears that applicants are more attuned to negative experiences than positive ones so even subtle negative cues may ruin an otherwise positive applicant experience. This may lead to negative outcomes such as a negative reputation for the organization or a reduce likelihood that the applicant will accept a job offer.
Takeaway 3: Technology needs a human in the loop which can amplify, reduce, or ignore human biases
When the presenter discussed this dichotomy, two concepts stuck out when thinking through how automated decision making can be combined with human judgement. The first concept was algorithm aversion, or the tendency for people to dislike when algorithms make decisions for them. Dr. Ryan further explained people are generally accepting when algorithms make routine decisions with minimal consequences. However, when major decisions are involved, such as the decision to hire someone into an organization, people are less likely to rely on information provided by an algorithm. The second concept related to this topic is the autonomy-validity dilemma, or the tradeoff between humans desire for autonomy when making decisions and the superiority of mechanical combinations of information when making decisions. As brought up during the question and answer portion of the talk, this mirrors findings in the employee selection literature that hiring managers have a strong desire for making autonomous decisions even when mechanical combinations of data may provide better prediction across a range of outcomes. Dr. Ryan referenced an NSF grant that she is contributing to called the Trapeze Project. The project is still in the early stages but aims to study many of these issues in detail so look out for future research in this area.
Thanks to all of you who have taken the time to read this blog and a special shoutout to those who attended the talk and contributed their thoughts. Finally, I’d like to personally thank Dr. Ryan for an insightful and engaging presentation! For others who attended the event, feel free to share your thoughts and takeaways in the comments below. I hope to see you at next year’s PTCMW Fall Event!
