The most popular tech related articles always contain an element of fear. Recent talk has centered on not who will have their jobs replaced by robots and computers, but when. Blue collar, white collar; it’s only a matter of time. To some degree this fear of technology has always been a part of our human condition. Dr. Frankenstein was killed by the monster he created, John Henry died after narrowly defeating a steam driven track laying machine. More recently, a computer beat the world champion at Go; a game so complex that there are more game possibilities than there are atoms in the observable universe.
By best estimates, we still have at least a decade or two until computers replace humans in jobs that are exceedingly complex and require higher order decision making or forecasting skills. Until your entire audit department is replaced by a single computer, you are still going to need people to fill those seats. Before we delve into the future of hiring, it’s worth discussing first why the traditional hiring methods you’re currently using are scientifically irrelevant.
Interviews: A Long Running Flop
As it turns out, the interview process has been only marginally improved since 1921 when Thomas Edison created a famously difficult written test to evaluate potential job candidate’s knowledge. While knowing where cork came from may be useful in certain circumstances, random recollection of facts is not. When the test was leaked, the New York Times berated the test, saying it was, “a test of a man’s memory and store of miscellaneous information, rather than his knowledge, reasoning power or intelligence.”
Having evolved since Edison, the interview has become a near ubiquitous hurdle in obtaining a job. As it turns out, multiple studies have shown that humans are hardwired to make sweeping assumptions and judgements within milliseconds of meeting an individual. First impressions will go on to influence the remainder of the interview, nearly regardless of the questions posed or the answers given. Typical unstructured interviews have been shown to utterly fail to predict future job performance in any meaningful way. Actually, they may hurt your firm’s chances of hiring the right person. Even well planned structured interviews, an impossible task for most firms given the sample size required, only marginally improve the chances of success.
Personality Tests: No Better Than a Coin Flip
More recently, personality tests have become an increasingly popular method to assess the suitability of a potential hire and attempt to limit interviewer bias. Incidentally, the basis for these tests also originate in 1921, but this time with Carl Jung, in his historic, “Psychological Types”.
In a 2006 Bloomsburg University paper, “Personality: What It Takes to Be An Accountant,” the authors analyzed a series of studies dealing with the personality types of accountants. Predominantly using the popular Myers Brigg’s test, most studies showed that CPAs tend to be either ESTJ (Extroverted, Sensing, Thinking, Judging) or ISTJ (Introverted, Sensing, Thinking, Judging).
While it would be tempting to simply administer a personality test to all potential hires, there is a major problem. Over the last thirty years, personality tests have been subject to scathing criticism by professional psychologists. Statistically, there is only a 50% chance that a potential hire will fall into the same personality category if they take the same test after a five-week interval. Furthermore, these tests are easily rigged by the applicant. They may consciously answer questions to shoehorn themselves into selectable traits such as those for organizational skills and communication proficiency.
A recent survey by CareerBuilder found that the Financial Services Industry reports the highest percentage of lies on a resume. It’s not beyond reason to assume that job candidates will do “whatever it takes” to land the job; including manipulating their personality test answers to meet your perceived requirements.
Even the publisher of the famous Myers-Briggs test, CPP Inc., admits in a roundabout way, that it’s not designed for hiring. CPP, Inc., the exclusive publisher of the Myers-Briggs test (MBTI) states, “It is unethical to use the MBTI for hiring…people of many different types excel at the same job for different reasons. Individuals should not be pigeonholed based on their personality preferences.”
Taken in total, the consequence is that you have roughly a 50/50 chance of hiring someone that falls within the personality parameter that you’re looking for. If they answer truthfully.
GPA: Less Relevant Than You Want to Believe
Being analytical, it makes sense that CPA firms would focus on GPA. In interviews with partners around the country, and through various online message boards we searched while researching this topic, we understand that there is a generally accepted minimum GPA to qualify for an interview. Under this threshold, generally a 3.4 GPA, most accounting firms won’t even offer an interview.
There are many issues to consider if you’re solely using a candidate’s GPA. First, you’re going to be competing with every other firm using the same metric. Furthermore, unless the candidate is coming from a local institution that you’re familiar with, academic rigor across the nation is anything but standard. All things being equal, would you rather hire a 2.9 GPA from Harvard who worked her way through college, or a 4.0 GPA from a state university?
GPA does not encompass the broad set of skills you’re looking to gain for your firm. Even the AICPA acknowledges this. In their Core Competency Framework, they list three main pillars that a student will need to enter the accounting profession. In their broad framework, the “Personal Competencies”, and the “Broad Business Perspectives Competencies” have seemingly no direct correlation to a grade point average. Named skills in these categories such as leadership, communication, critical thinking, and marketing, don’t readily associate themselves to GPA.
In a recent Business Insider Article, Josh Bersin, the principal and founder of Bersin by Deloitte, an HR consultancy, spoke on the issue of hiring based upon GPA. “Companies want someone who thrives on challenge [and is] willing to learn something new…[They want] a seeker of information, willing to adapt. If you’re the type of person that wants to be told what to do, you might be a straight-A student. In fact, you might even be a better student than the other type of person.”
Although this may seem like a radical notion, the idea that GPA fails to correlate with success has been around for quite some time. In the 1989 paper, College Grade Point Average as a Predictor of Adult Success: A Meta-Analytic Review and Some Additional Evidence, the author comes to a similar conclusion. “ [I]f a relationship does exist between GPA and job success it is tenuous at best…the use of GPA in this capacity is not recommended.”
What if you use a combination of every traditional method to assess a potential hire?
Laszlo Bock, the former head of hiring at Google explained how generally ineffective this would be. In a recent Wired article, he explains that unstructured interviews only account for 14% of an employee’s performance, reference checks 7% of performance, work experience 3%, work samples 29%. Taken in perfect totality, that still only accounts for roughly 50%; or another coin toss.
Ultimately, you’re likely looking for two seemingly basic, but profound, criteria. Will they be good at their job, and will they work hard? These two questions form the basis for every HR department’s quest to advise management on hiring qualified individuals.
Will They be “good” at their job?
To answer this most beguiling of questions, a company called Knack alleges to have found an answer. According to Guy Halfteck, founder and CEO, “Knack is an audacious technology startup that helps people from all walks of life discover their hidden human potential. Knack gives people a mobile gateway to economic opportunity and a pathway to fulfill their aspiration for a better life. At the same time, Knack enables schools, colleges, and employers to unlock and realize the hidden potential of students, learners, jobseekers and employees.”
That’s a bold statement. Does it live up to the hype?
Knack is a series of three smartphone games, all of which are free to play. The general idea behind Knack is to combine, “machine-intelligent data analytics” to revolutionize the way firms assess the suitability and aptitude of potential hires.
In this game you serve sushi to hungry guests based on their mood
Here, you launch water balloons at flowers
Referring to the AICPA’s Core Competency Framework, Knack alleges to measure approximately nine of the 19 suggested competencies directly. A few more could easily be correlated for your purposes. More importantly, not only can predict if they’d be suited to the accounting profession, but the software can also lead you to hire for auditing, consulting, and other specialty areas within your firm, via their “superknacks”.
Is Knack Accurate?
It is worth noting that the thousands of split second decisions necessary would make “gaming” the system incredibly difficult, and there is no discernable concept of how certain gameplay decisions could lead to results that would be beneficial to the interviewee.
The authors played the available games to determine how precise the predictions could be. Upon completion, the authors were ranked from 1-5 stars based upon specific traits, as deciphered by the software. They were given a ranking on several attributes such as executive presence, critical thinking, and teamwork; all potential “rainmaker” attributes sought after by partners, but heretofore unmeasurable in any statistically meaningful way. The app also displayed fields in which those attributes would most be best utilized.
While not a statistically viable sample size, the app’s accuracy was uncanny. One author seriously contemplated medical school when he left the Navy. While he’s currently in sales that specializes in accounting firms, he has a bachelor’s degree in Systems Engineering. Furthermore, he reads voraciously on the topic of economics, and had recently finished a book on the history of the Federal Reserve before writing this article.
Recently Knack was used to assess the ability for orthopedic surgery resident training screening processes. Much like CPA designation hopefuls, medical residents require years of training and oversight. In the end, they are required to pass a series of tests, including the notoriously difficult Orthopedic In-Training Exam (OITE). In the peer reviewed paper, “Can video game dynamics identify orthopedic surgery residents who will succeed in training?” the authors came to a remarkable conclusion:
” Despite having a large sample of intelligent, highly motivated applicants to choose from, orthopedic residency programs still encounter trainees whose performance is considered subpar.” Using Knack, “A predictive validity was identified for those who scored higher on the OITE. Multiple game-based features (heuristics, flexibility, social perception, attentiveness, processing speed, memory, adaptive and context dependent processing speed and adaptive flexibility) contributed to the prediction of better performance on the OITE exam.” Using Knack, “A predictive validity for higher performers on the OITE combined with all clinical skills was also obtained. This model significantly predicted resident success.”
In short, they took a pool of already intelligent candidates and determined who was most likely to succeed in the program; much like accounting graduates who must pass the CPA exam.
Many firms have dealt with staff accountants who have struggled to pass the CPA exam. Time and resource intensive measures, such as carving out otherwise billable hours and mentoring the staff member to assist with exam preparations are costly. Implementing a method of screening such as Knack may significantly alleviate these future worries for partners and staff members; freeing their time for other pursuits.
So, does it really live up to the hype? The authors believe it does.
Of course, these games are free for personal use, so you can always download them yourself. There are also specific options available for businesses, with additional features if you’re considering this technology for your firm. Having another tool by which to understand and entice a new prospect is never a bad idea.
Furthermore, advances in the concept of neuroplasticity lead us to believe that games of this type could have long term use, as personality changes with age. It may be used over the course of a CPA’s career to best steer them towards personally suitable projects and positions that best align their strengths and weaknesses with the firm’s objectives.
Lest you believe your firm will laugh you out the door when you broach the idea of using a video-game for your next round of hiring, know that you will be in esteemed company. Here are some of the high-profile companies reportedly using Knack’s technology: Salesforce, Tata, Citi Group, American Express, Daimler, GE, Cigna, and Bosch.
While the space for these types of hiring aids are currently limited, at least one other company, Pymetrics, alleges much of the same results.
But, Will they work “hard”?
We all have staff or friends that, somehow, seem to accomplish more than the average human, every week for years on end. Is this simply an inevitable outcome of “passion” within their field of work, or an innate characteristic of that individual?
Interestingly, the output work capacity for humans is not a linear relationship, but exponential curve. Paretto’s Principle, more popularly known as, “The 80/20 Rule”, has been shown across widely varying domains since the late 1800’s that 80% of any given output is achieved with 20% of the inputs. In relation to your firm 20% of your employees likely produce 80% of the work output.
Put generally, your hardest working staff members produce a disproportionally large amount of work. Undoubtedly you can name at least a few individuals in your firm that seem to produce far above their peers.
In the world of psychology, this characteristic of productivity is known as “Conscientiousness.” It is a direct measure of, industriousness, self-discipline, and orderliness; a valuable trait in accounting. Given a choice, you’d want to hire those who will dutifully outwork most of their peers; i.e. those with higher conscientiousness. That payoff can be a serious economic motivator.
In his paper, Basic Psychometric Issues Surrounding Performance Prediction, psychologist Jordan Peterson relates how valuable it would be for a firm to screen for even incremental advances in employee conscientiousness. Using a formula derived by Hunter and Hunter in their study, Validity and Utility of Alternative Predictors of Job Performance, Peterson shows that a firm with only 10 people screened for conscientiousness would outperform their peer group by an additional $3,500,000 over a 5-year period.
Put another way, they could generate an additional $70,000 in revenue, per employee, every singly year.
A seemingly attractive alternative to screening candidates would be to hire as many people as possible and then weed out the non-performers over time. Certainly, this sounds analogous to the internal hiring process of four very well-known accounting firms. Unfortunately, Price’s Law directly refutes this idea. Price observed, in basic terms, that there was an unequal distribution of productivity in most domains. The square root of the number of employees will produce half of all output in your firm. Ergo, if you have 100 employees, 10 of them will produce 50% of the work.
In relation to your firm, Peterson states, “Price’s Law means that a relatively small number of people will allow [you to], massively outperform the large majority of competitors. Thus, selecting higher up on the performance or ability distribution produces unexpectedly large economic gains.”
While assuredly not a perfect measure, Peterson states that an appropriate test for conscientiousness can change the ability for you to hire a “hard worker” from 50-50 to 67.5 percent in your favor.
Testing for this type of trait is relatively straightforward in the “Big Five Personality Test”. While they may appear similar, it should be noted that the Myers-Briggs test we discussed earlier is theory driven, whereas the Big Five is data driven. In stark contrast the Myers-Briggs Test (MBTI) the Big Five Personality Test has high scientific validity, and reliability, in predicting job performance and team effectiveness. Furthermore, it has been shown to be valid across varying cultures and timeframes within industrialized, literate, societies
It’s Great for Both Parties
It’s worth noting that hiring the best person for the job isn’t purely in the employer’s best interest. No matter who is ultimately hired, there will be long hours, difficult projects, and mistakes made. However, an employee that works hard at a job they love can feel a profound sense of satisfaction and connectedness with their work.
Energy Project, a firm focused on employee engagement, recently conducted a study on workplace fulfillment in partnership with the Harvard Business Review. “Employees who derive meaning and significance from their work were more than three times as likely to stay with the organization…They also reported 1.7 times higher job satisfaction and were 1.4 times more engaged at work.”
This of course has a direct impact on how profitable your firm can be. Companies where employees volunteered for extra work had a 14 percent operating margin. Those with highly engaged employees nearly doubled that operating margin at 27 percent.
Ultimately this could provide for more engaged employees who are also better compensated than their peers are competing organizations. It’s not a stretch to believe that this could provide a successful feedback mechanism within your firm that could prove beneficial for decades.
Are They Legal?
The hiring process can be fraught with concerns for employers. Of all the employment related laws, two are particularly relevant to this discussion.
Title VII of the Federal Civil Rights Act of 1964 generally prohibits discrimination based upon, “race, color, religion, sex, or national origin” in the process of selecting employees. However, the use of a “professional developed ability test” is not prohibited unless it is, “designed, intended or used to discriminate because of race color, religion, sex, or national origin.”
The Americans with Disabilities Act (ADA) prohibits employers from conducting medical examination pre-employment. In certain cases, various questions may be considered a medical exam if they could reveal a mental illness.
For both the Big Five Personality Test, and Knack, it’s difficult to see why they would not be allowed. To date, the authors were unable to find evidence of either test being considered discriminatory under the guises of Title VII or the ADA. However, the authors are not providing legal advice; we are not lawyers. You should always seek advice from your Employment Practices Liability Insurer, HR professional, and competent legal counsel, for all aspects of your hiring and recruitment processes.
Given the available evidence, it appears that traditional methods of hiring are firmly rooted in tradition as opposed to scientific validity. Administering a combination of the Big Five Personality test, and occupational specialty games such as Knack within your firm, may provide significant economic and social benefits for all parties. There’s no time like the present to upgrade your hiring process from an art, to a science.
—Joseph Brunsman (joseph@cplbrokers.com) is the vice president of Chesapeake Professional Liability Brokers in Annapolis, Md. He is a co-author of True Course: The Definitive Guide for CPA Practice Insurance.
Dan Hudson (dhudson@cplbrokers.com) is the president of Chesapeake Professional Liability Brokers in Annapolis, Md. He is a co-author of True Course: The Definitive Guide for CPA Practice Insurance.
Byron Patrick (bpatrick@networkalliance.com) is managing director of CPA practices for Network Alliance Inc. in Reston, Va.
Thanks for reading CPA Practice Advisor!
Subscribe Already registered? Log In
Need more information? Read the FAQs
Tags: Firm Management, Payroll, Staffing, Technology
