Compliance

Big Data Can Be a Great Hiring Tool — But It Isn’t Foolproof

October 14, 2020 by Michael Callen

Big data’s moment has arrived, and it’s already redefining business processes, overhauling decision-making, and making prognostications for the future in hiring. Although big data promises to give recruiters newer and better tools to find the right applicants, it could also trigger new ethical questions.

Big data can be “the great equalizer” because it doesn’t care about what college someone went to or who their parents are; it also doesn’t factor in gender, race, or disability status. Whereas companies once depended on gut feelings about applicants in their hiring decisions, they can now use artificial intelligence (AI) and analytics to parse the data on current employees and apply the results to prospective employees in a pool of candidates.

In theory, it’s a great solution. But in practice, it can introduce unintentional and harmful bias into the hiring process.

No One-Size-Fits-All Approach

Using algorithms to help determine hiring decisions comes with the risk that the datasets they’re built on lack the necessary information — and could produce false results. More than that, the systems could be poorly designed or outdated.

All of this is to say that employers should be careful when integrating big data into their hiring processes. Data has terrific potential, but it isn’t a foolproof tool. With this in mind, we must find ways to use this tool in pre-employment testing while keeping the potential big challenges of using big data in mind.

The most important thing to remember is not to go all-in without understanding the risks. Things can go wrong quickly if HR departments aren’t mindful. For instance, geolocating applicants with big data can be an incredibly valuable tool in candidate selection. But businesses that primarily do this in the suburbs, far away from city centers, can quickly whitewash what could have been a diverse workforce. Removing bias from AI may not be an immediate option, but it’s crucial to be mindful of the ramifications that bias could create.

Many projects to develop AI based on big data have resulted in biased and discriminatory algorithms. Why? The answer lies in the algorithm creation. When the people who construct algorithms and evaluate data come from similar backgrounds, their personal experiences and biases can influence their work. An algorithm’s results can only be as good as the instructions used to build that algorithm, so developers must account for potential implicit bias in AI algorithms.

With all this in mind, here are three specific things HR teams must know to reap the benefits of using big data responsibly moving forward:

1. The more selection depends on data, the more analysis is necessary to prevent adverse impact.

While the Uniform Guidelines on Employee Selection Procedures allow validity to trump adverse impact, Section 15(C)6 states that alternate selection procedures should be investigated and analyzed in light of their impact, describing the scope, method, and findings. This is why content validity is so important in pre-employment testing. If an employer can say it is simulating essential tasks from the job in its testing and can justify the practice of those in the test, then it is protected from adverse impact; but if all it can say is that it analyzed data based on which applicants preferred one color to another — or some other big data construct — the risk is still a serious threat.

2. Work with vendors that focus on diversity.

Be intentional about working with data companies that have an excellent track record on diversity. Data on its own isn’t enough to prevent bias in the hiring process, so it’s essential to have a partner and solution that will work together with you for the good of your efforts. You should also consider what any vendors are doing to combat bias, paying close attention to what you find. If they have not been proactive on this front, it might be time to find a new partner.

3. Don’t set it and forget it.

Finally, work with your technical colleagues to ensure any data vendors understand what kinds of interventions could be necessary to address algorithmic bias. Conscientious vendors share validation results for their algorithms, highlighting whether their results introduce any bias — and conscientious HR leaders ensure they’re implementing technology that has been and will continue to be updated.

Big data will likely continue to play an influential role in the future of every industry, and HR is no exception. That data can be incredibly useful, but it doesn’t come without risks. As an employer, be mindful of the potential pitfalls and intentional about avoiding them to keep bias in AI algorithmsout of your hiring process.

Do you want to learn more about how to use big data in your hiring processes? Contact us today!

Criterion Validity and Pre-Employment Testing

October 1, 2020 by Michael Callen

Skill and ability tests should accurately reflect a business’s needs and the critical duties required on the job. Ensuring that this is the case is known as test validation.

There are many reasons employers should make a point only to administer tests specifically validated for positions in their organization. Doing so ensures that candidate selection decisions are effective (utility), but it also provides a defense that employers have justified the specific testing in their hiring processes (defensibility).

The Differences Between Content and Criterion Validity

When developing hiring assessments, there are two main types of validity: content and criterion.

Content validity shows the connection between the test, the skills or abilities necessary to perform well on the test, and the duties required to succeed on the job. Essentially, it indicates whether candidates can perform essential job duties. While this is useful information, tests are capable of so much more.

Meanwhile, criterion validity goes beyond mere ability by statistically predicting success in certain job performance areas. In many ways, it takes content validity to the next level.

It’s important to note that criterion validity does not always exist — and it isn’t always necessary. Knowing that a person can type, for instance, is important when hiring a new member of your office administrative staff. That said, the ability to type is not necessarily predictive of overall success in that role.

Sometimes, though, one can use test scores to look into the future and predict how likely candidates are to be successful. Criteria such as job performance ratings, customer satisfaction, conflict management, training time, and even attendance can often be correlated to scores. In these cases, criterion validity is a worthwhile consideration when crafting a selection plan.

How to Establish Criterion Validity

Employers looking to improve their pre-employment testing and determine whether it’s capable of tracking criterion validity should consider auditing their current assessments and comparing them to known performance metrics. Fortunately, establishing this test validity only involves a few steps:

1. Identify key performance dimensions. Start by determining which key parameters and performance dimensions are essential to the job. Consider all critical, job-related aspects — especially those that differentiate great performers from the rest.

2. Ask current employees to take an assessment. Have incumbent employees who currently work in the role for which you are hiring complete the assessment you will use with potential hires. Use a broad range of performers (from underachievers to high-performers) to learn as much as you can from the analysis.

3. See what statistical relationships exist. Correlate test scores to different job performance areas using Excel or statistics software like SPSS. A correlation of 0.20 or greater — with a 95% or better certainty — is considered to be statistically significant.

If you can follow the above steps and find a correlation between your testing and job performance metrics, you have established criterion validity.

How TestGenius Can Help

The TestGenius platform is perfect for conducting pre-employment testing with incumbent employees (for establishing test validity) and job applicants.

TestGenius works with employers to develop more effective testing solutions to help them gain greater utility from their assessments. We are also currently seeking partners for ongoing validation studies in the retail and emergency services markets. If you’re interested in participating in a similar study in these markets, or if you would like to propose beginning a new study in your own space, please contact our team.

How to Use Preemployment Tests and Mitigate Adverse Impact Along the Way

May 7, 2020 by Michael Callen

Plenty of recruitment and hiring professionals rely on applicant testing during the prescreening process. Assessments not only help narrow the field, but also shine a light on superstar performers.

Unfortunately, even the most professionally developed tests can impact members of certain groups differently. If that happens, the test is said to have an “adverse impact” on the negatively impacted group — turning the assessment test from a corporate asset into (potentially) an expensive liability.

What Is Adverse Impact?

Adverse impact refers to a procedure, practice, or test that consistently registers a substantially different passing rate between two groups, regardless of intent.

Many equate adverse impact to discrimination. This is not true. You will almost always be able to find adverse impact because data can be analyzed in any number of ways. But the Uniform Guidelines on Employee Selection Procedures (UGESP) makes it clear that if a test potentially has an adverse impact, a rigorous validation process can prove the test is “valid and consistent with business necessity.”

For example, we often find that computer tests have an adverse impact on applicants from older generations. So many of the input devices that are necessary today (e.g., keyboards, mice, etc.) did not exist just a few decades ago. For most jobs, using a computer keyboard and mouse to enter data into specific fields is “valid and consistent with business necessity.”

Another adverse impact example would be physical ability testing. Women fail firefighter physical exams at a rate of 50-to-1, which is some serious adverse impact. That said, being able to drag a dummy to safety is necessary, so the adverse impact does not prevent the physical ability test from being used.

Some organizations have gotten into trouble because they tested for above-and-beyond job requirements. Imagine a job for which a worker has to carry 50-pound bags of concrete and stack them on pallets. If an organization tests with 95-pound bags — when bags on the job weigh no more than 50 pounds — the test would be deemed invalid, and the employer would be responsible for the adverse impact.

A couple of years ago, CSX Transportation paid $3.2 million to settle a sex discrimination lawsuit when its physical abilities test unfairly had an adverse impact on female job candidates. Per the U.S. Equal Employment Opportunity Commission’s (EEOC) allegations, CSX conducted isokinetic strength testing, a three-minute aerobic test, and arm endurance tests that had a discriminatory impact on women applying for jobs as conductors, material handlers, and clerks. The EEOC did not claim that the discrimination was intentional.

How to Mitigate Adverse Impact

Of course, skills-based tests aren’t always airtight, particularly if they’ve been developed in-house. Businesses that want to give all candidates the greatest chance of showing their abilities and acumen should focus on skills-based tests that show maximum utility at the least legal exposure risk.

Plus, leaders can go one step further by requesting or issuing a validity report. Software from TestGenius and CritiCall includes a content validation wizard that follows sections 14C and 15C of the UGESP; our software queries successful incumbent employees for specific information that builds a UGESP 15C validation report. The employer is then justified in using the test in a way that is “valid and consistent with business necessity.”

Furthermore, our validation process helps to hire better performers because it matches the test to products created on the job using skills necessary on day one. That makes it a win-win for the employer.