Pre-employment Simulation Tests: They're in Demand and "On Demand"

1 Jul, 2007

By: Linda Driscoll-Dobel

---

Contact center agents are your company to your customers. Agents define your customers’ image of your company. Any bad hire creates a ripple effect that damages your organization for years. Competitive organizations cannot afford to employ agents who have contact center skill deficiencies to act in the role of responding to their lifeblood, the customer. Yet, the process of hiring contact center staff has traditionally been fraught with inefficiencies, inconsistencies and, too often, poor results in terms of immediate and long-term performance of those hired.

There are a number of reasons for this unfortunate situation, most notably the faulty methods engaged to facilitate the hiring process. Specifically, many organizations have relied on pencil-and-paper tests and broad-based general job interview techniques that do not drill deep enough into the realities of contact center work.

Some organizations rely on training to achieve their success. Training is absolutely critical, yet the gap between current ability and success as a result of training is often more costly and time-consuming than some organizations understand. How many customers can any organization afford to use as experiments?

Experimentation and failure are extremely costly. Fortunately, most of the reasons call center agents fail are well known. While the reasons are straightforward, it is not so simple to detect these possible faults before hiring an employee.

Just a few of the reasons contact center agents fail include:

• They’re rude to customers
• Inattentive listening
• Inability to follow instructions
• Providing customers with incorrect information
• Inability to manage the workstation and the customer discussion at the same time (multi-tasking)
• A chronic habit of making mistakes when entering critical customer information
• Missed opportunities to recommend products or services that benefit customers.

Predicting in advance if someone’s performance will suffer from these shortcomings is improbable with a written test. Finding employees whose skills take the correct path in these areas cannot be discerned on paper or through an interview. After all, written text and real life are two different worlds.

Then, how can it be determined in advance who will be a successful contact center representative and who will not? Intuitively, the best way to find out would be through direct observation --- watch and listen to the potential reps as they take calls from customers. But customers are far too valuable to use as “test cases” to find out if someone can do the job. So what’s the solution?

One answer that has proven effective is simulation. Twenty years ago, a company named Employment Technologies Corporation (ETC) hit upon this idea and pioneered and established a testing solution now known as EASy® Simulation Assessments. To develop an EASy Simulation for contact center representatives, the company worked with 29 diverse call centers to gain a realistic view of the job and the necessary abilities to perform the job. The work by ETC included a blind study with 800 agents to compare predictions from the simulation with actual job performance. Time after time the prediction results were outstandingly accurate and resulted in hiring individuals who allowed the companies to enjoy improved employee performance, decreased training requirements and increased retention.

Employment Technologies Corporation has now taken its technology a step further. Staying in pace with the current state of the industry, it offers Web-delivered (remote) simulation for organizations that desire to cast an even broader net when it comes to making the-all important hiring decision for contact center associates. It is a natural extension of the proliferation of online job postings in the virtual marketplace and follows the trend of fewer on-site conferences and meetings as well as the trend toward off-site employment. The ETC Web-delivered product is called EASy Simulation On Demand. Web-delivered simulation for hiring saves the company the expense of bringing candidates in for a face-to-face interview before it has been determined whether or not the person is a fit for the job and it saves the candidate all of the time and effort it takes to go to an onsite venue prematurely. It enables organizations to evaluate applicants remotely from any computer that has Internet access. In addition to extending the process to individuals globally, it has proven a valuable tool at job fairs and can, of course, be used at the corporate site as well.

While every organization will have unique reasons for desiring a product such as EASy Simulation On Demand, the immediate and most obvious advantages include:

• ETC’s Web portal for 24/7 global access
• Realism of simulation, real-time delivery, with real-time results
• Expanded access to a larger recruiting pool of qualified applicants
• Elimination of scheduling and on-site testing, if desired
• Access to pre-qualify applicants as needed, when needed, with controlled applicant flow
• Elimination of IT barriers associated with software installation
• Reduced administrative overhead.

The fact that EASy Simulation On Demand emulates real-life interactions between the applicants and “virtual customers” really sets it apart from other hiring methodologies and gives applicants a clear idea of whether or not contact center work is appealing to them --- without going live with actual customers and potentially jeopardizing customer relationships. It can also show applicants they may have talent for a job they didn’t know they could master. In addition, it eliminates the “stretching of the truth” or “faking” often found on personality tests and resumes. With simulation, people don’t fake responses; they respond to questions intuitively and therefore a more effective picture of the applicants can be drawn.

The product doesn’t stop there, though. EASy Simulation On Demand also goes a step further and provides an invaluable profile of candidates, which can really help to eliminate the scourge of the training maze.

The software provides agents with a map for personal development before they enter the training maze. With their own personal development plan, each agent significantly increases his or her learning, and success is achieved with considerably less effort. EASy Simulation On Demand goes right to the heart of this concept by providing a series of feedback reports and drill-down capability that assist in making hiring decisions as well as identifying both individual and group developmental needs.

 

Does it Really Work?
A study to evaluate the effectiveness of ETC’s EASy Call Center Simulation (EASy CCS) was requested by a major insurance provider to determine if the simulation could improve its current selection process and what, if any, benefits the simulation might be able to provide.

Applicants from four different client locations participated in a “blind” predictive study. Applicants’ simulation scores were collected and tracked for six months.

Study results clearly demonstrated test scores accurately predicted key measures of job success for the insurer’s claims service representatives. Applicants who scored high on the simulation were 35 percent more likely to meet expectations for overall quality score than those who scored low. Applicants who scored high on EASy CCS were 54 percent less likely to be involuntarily terminated compared to those who scored low.

No adverse impact was evident from the use of the selection tool for any race or gender subgroup.

Clearly, the inclusion of the simulation in the associate selection process will allow the company to make better hiring decisions and improve overall performance on the job.

 

Modern Hiring Procedures Swing into Financial Action
One of the great things about Employment Technology Corporation’s EASy Simulation tools is that they are effective across industries. Early adopters of the technology are financial institutions. One financial institution began using ETC’s EASy Call Center Simulation (EASy CCS) in March, 2006 as part of its associate selection process. Although they recognized that EASy CCS is a fully validated, multimedia assessment tool used to select contact center agents, the large financial firm wanted to further establish the validity and legal defensibility of the test within its specific environment. Of course, as a matter of protocol, prior to using the test, the financial institution had completed ETC’s standard job-match process for establishing the appropriateness of using EASy CCS for selection of call center associates.

To evaluate the effectiveness of the simulation, data were collected from 16 hiring locations between April, 2006 and October, 2006. Applicants were hired using the financial institution’s established selection process, with the addition of EASy CCS. Applicants with EASy CCS scores lower than the minimum cutoff were not hired. The institution provided performance and retention data for the hired candidates from May, 2006 to December, 2006 to ETC, which then examined the relationship between associates’ EASy CCS test scores and their subsequent performance and retention.

Study results clearly demonstrated that test scores accurately predicted key measures of job success for the financial institution’s call center associates. Measures of associate performance, call quality and retention were seen to correlate well to EASy CCS test scores. The results are both conclusive and compelling, as follows:

Higher EASy CCS scores were significantly correlated with higher associate overall job performance ratings (p < .01)

Higher EASy CCS scores were significantly correlated with higher call quality monitoring scores (p < .01)

Associates who were still employed after 90 days had significantly higher EASy CCS scores than associates who had terminated.

After adding EASy CCS to the selection process, the client’s 90-day attrition rate dropped by 44 percent in the following year.

A fairness analysis demonstrated that EASy CCS is a fair and valid predictor for gender and race subgroups.

This study among many prior studies shows consistently positive results from using EASy Simulations. It enabled the firm to further establish the validity, legal defensibility and the bottom-line value of the test within its environment.

Further, consistent results show that the EASy Simulation provides an accurate prediction of an applicant’s future job performance as a call center associate. The tool has helped the firm avoid the high costs associated with hiring and training individuals who are likely to leave or perform poorly on the job. As EASy CCS continues to be used, its call centers will become staffed with more high-quality agents who have outstanding customer service skills.

 

My Reality Test
In early spring, I had the opportunity to take ETC’s EASy Simulation On Demand. To be honest, before I took the simulation, I thought it would be a piece of cake and that I would, without a doubt, be one of the most highly qualified candidates they ever evaluated. Wrong, wrong, wrong.

Here’s how it went.
I received a link and password from ETC (as if it were a company I was applying to for a job). I was told I had a certain amount of days to log in and perform the simulation. It was made clear to me that the test was timed (45 minutes is the allotted time frame), so I would need to set aside a quiet time that would not be interrupted. I was also instructed that I could not enlist the help of anyone else to perform the simulation. It was stated that should another person perform or help with the simulation, follow-up tests would reveal the deception.

The goal of the test sounded beneficial to both my “prospective employer” and me --- by taking the simulation, I could determine if contact center work was for me, and the employer could determine if I warranted follow-up interviews, which could possibly include traditional screenings such as the standard phone interviews and in-person interviews. Obviously, a negative determination to continue the hiring process with me should I not meet their criteria could significantly reduce all the negatives associated with turnover, both for the employer and me.

As I began the simulation, my identity was verified and I was again prompted to ensure I had uninterrupted time to perform the simulation. Then I began the simulation and that’s when the “fun” started. My task was to “answer” calls by clicking on buttons that lit up when a “call” came in for one of three diverse “clients” and be prepared to enter account numbers, listen for verbally simulated details exchanged between the “pretend callers” and “me” (my role was simulated as well. In other words, I didn’t use my voice to actually speak, just computer input) and decide and input what I thought would be the most appropriate answers to the widely varying inquires. It was incredible how “real” the simulation felt.

In retrospect, I describe the experience as a fun challenge that really made me think and realize my true potential as a contact center rep. While I really wanted to provide the best answers, I was definitely challenged as I found I needed to ask the “callers” to repeat information too many times to be acceptable to anyone looking for assistance from a contact center representative, and I was seriously deficient at entering account numbers and addresses. The time it took to complete the simulation seemed to go by very quickly, so I can imagine that must often be the case for real contact center agents. The simulation also affirmed that contact center jobs require a variety of skills --- skills that I don’t possess. I guess I will stick to being an editor for now, besides, based on my results, I don’t think a contact center would hire me to be on their frontlines.