Category Rating – Does It Work?

View this article online at http://www.fedsmith.com/2016/07/10/category-rating-does-it-work/
By • July 10, 2016 Comments

Image of 3d person standing next to category rating scale

In my most recent blog post I said part of the current difficulties agencies are experiencing with hiring non-veterans is caused by how they have implemented category rating and not by veteran preference itself. I promised to address category rating in a future post. This is that (long) post.

When I talk to federal employees, hiring managers, and veterans about federal hiring practices, it is common for them to have little understanding of category rating and how it works. Unfortunately, I have also found some HR folks who do not understand how it is supposed to work. So – I am going to try to demystify it a bit and show how it is supposed to work and how it sometimes works badly.

To understand category rating we need to understand the step-by-step process for reviewing applications and then go back in time and understand category rating’s precursor, the “rule of three.” For many years the rule of three was gospel when it came to federal hiring. Agencies rated applicants and assigned a numerical score on a hundred point scale. Really, however, the scale was only 30 points, because any qualified applicant got a score of 70. So the basic score range was 70 – 100. But this is the government, so simple is simply not good enough.

The Process

Once an agency has applications for a job, it has to decide who to refer on a certificate (the document listing the candidates a selecting official may choose from). That begins with determining basic qualifications. Every job has qualifications requirements that are laid out in OPM qualification standards. They are usually written in the form of a number of years of experience, education that is required for the job, and combinations of experience and education that would qualify an applicant. You find them in the beginning of most applicant questionnaires in the form of a question or two. An applicant who does not meet the basic qualifications requirements is determined to be not qualifiedand is out. Scores and veteran preference are not a factor at this stage of the process. It is important to note that HR and subject-matter experts (SMEs) are supposed to play a big role in this process. Regardless of how an applicant answers the qualifications questions, HR (with advice from SMEs) is supposed to review the resume and make a qualifications determination. For example, if an applicant says “Yes” to the question regarding specialized experience, but has no such experience identified in his/her resume, HR is supposed to override the answer and rate the applicant as not qualified. The systems HR folks use to process applications, such as Monster, Acendre or USA Staffing, are all designed to allow and document such changes.

Some jobs have “Selective Factors” –  qualifications that are also important for the job and are already required when the person starts the job. They typically require extensive training or experience to develop, are essential for successful performance on the job, and cannot be learned on the job in a “reasonable” amount of time. An applicant who does not meet a selective factor is ineligible and is screened out. That is why selective factors are sometimes referred to as “screen out elements.” As with qualifications requirements, HR is supposed to verify that the application reflects the required experience and override it if the applicant claims experience that the resume doe not reflect.

Based upon conversations with a lot of hiring managers in different agencies, I believe the number one problem agencies are experiencing now is that a thorough review of applications by humans is just not happening. HR folks are erring on the side of passing through to the next phase everyone who says s/he is qualified. Some HR folks tell me they do not have the time to do good reviews. Others say the rating process will wash out any unqualified candidates. I have even had HR folks tell me they believe it is illegal for HR to override what an applicant says on a job application. As you will see below, failure to do proper qualifications screening is a problem.

Once HR has completed screening applicants to ensure they are qualified, the process moves on to the actual ratings. The old process of the rule of three worked this way. Based on applicant responses to questions or reviewing their applications, HR assigned a score. That score has additional points added for veterans. Here are the categories of veterans and the extra points they earn:

  • CPS: 10-Point 30 Percent Compensable Disability Preference based on a service-connected disability of 30% or more
  • CP: 10-Point Compensable Disability Preference based on a service-connected disability of 10% or more, but less than 30%
  • XP: 10-Point Disability Preference; granted to recipients of the Purple Heart, persons with a non-compensable service-connected disability (less than 10%)
  • XP: 10-Point Derived Preference granted to widow/widower or mother of a deceased veteran, or spouse or mother of a disabled veteran;
  • TP: 5-point preference

Veteran preference points are added to scores, then they are put in order to determine who would be referred to the selecting official. This is where it gets a little complicated. The list on the left is a sample list of applicants and their raw scores + preference points. The list on the right is the same applicants with the scores put in the order they would go on a referral list.

Image of charts showing veterans preference scores

 

With the rule of three, the hiring manager would get a list that includes the top 3 candidates on the right. Jon Snow is well qualified, with a basic score of 88 points and an additional 10 for being a compensably disabled veteran (below 30%). Fred Smith, the next candidate on the list, meets the basic qualifications but no more. He is near the top of the list because he is a 30% disabled veteran. The most qualified candidate in this example is Jim Eliot. He has a perfect score. In this example, the hiring manager will get a list that includes the top 3, but s/he cannot select Jim Eliot because doing so would require passing over a veteran (Jon or Fred).

There are many more complexities in the rule of three, such as the three passovers and you are out rule when multiple jobs are being filled from the same list, but getting into those would take far too much time. If you are interested in knowing about that one, go to “Auditing a Certificate of Eligibles Under the Traditional “Rule of Three” Procedures” in OPM’s Delegating Examining Operations Handbook. Suffice it to say that the rule of three was hard to administer and drove HR, hiring managers and applicants crazy. Category rating was intended to make it much less complex.

Category Rating

Category rating was authorized by the Chief Human Capital Officers Act of 2002. It says “The Office [of Personnel Management] …may establish category rating systems for evaluating applicants for positions in the competitive service, under 2 or more quality categories based on merit consistent with regulations prescribed by the Office of Personnel Management, rather than assigned individual numerical ratings. Within each quality category established under subsection (a), preference-eligibles shall be listed ahead of individuals who are not preference eligibles. For other than scientific and professional positions at GS–9 of the General Schedule (equivalent or higher), qualified preference-eligibles who have a compensable service-connected disability of 10 percent or more shall be listed in the highest quality category.”

Category rating was an optional procedure that many agencies chose not use, until President Obama’s 2010 hiring reform mandated its use instead of the rule of three. OPM’s guidance on category rating gives agencies a lot of flexibility in how they implement it. OPM also published a briefing and a set of Q&A that explain the process in more detail.

Agencies are required to have at least 2 categories, although many use 3 or 4. Applicants are put into categories and then listed by category in preference order. Although the statutory requirements for category rating do not include assigning numerical scores, many agencies use scores because they are easier to administer. Examples of some agency policies are herehere and here. When agencies assign numerical scores, extra points are not added for veterans.

Image of chart showing example of veterans preference category ratingThe sample list of applicants from above looks very different in a representative category rating scheme. Where before we had 3 candidates on the list, but only 2 who could be selected (Jon Snow and Fred Smith), now we have 6 in the top category, only 4 of whom could be selected. Even though Jim and Betty have high scores, they cannot be selected because doing so would require passing over qualified veterans. From the perspective of a hiring manager, the result could be either bad or good. Bad because s/he still cannot select the non-vet with a perfect score. Good because 2 veterans with high scores who were not within reach before are now avail be to be selected. From the perspective of the veterans on the list, it can also be bad or good. Bad if you are Jon or Fred, because now there are 2 better qualified veterans who are more likely to be selected. Good from the perspective of George and Steve, because now they are within reach and can be selected. It is still bad for Jim, because there is no way he is going to get this job, regardless of how the rating process is done. That is not a change – he would not get the job under the rule of three and he will not get it under category rating.

Is Category Rating an Improvement?

Category rating was supposed to eliminate the confusion caused by the rule of three and provide agencies with a more flexible rating process that gave hiring managers more candidates to consider. For the most part, it does that. Some people believed category rating would reduce the impact of veteran preference. It did not do that. In fact, unless the top category is very narrow (e.g., 98 to 100 points), the impact of veteran preference was probably amplified. When agencies use broad ranges for the top category (e.g., 90 – 100 points), hiring managers are seeing more candidates, but all of them are veterans. When they fail to do good qualifications screening, the result can be that unqualified applicants who are compensably disabled veterans get put into the rating process when they should have been screened out. Then they float to the top, and the process breaks down.

When category rating was implemented, most people did not know we would be in 2 wars for many years, and that those wars would produce a high number of disabled veterans (1.2 million, or 1 in 3, veterans of Iraq and Afghanistan are disabled in some way). A total of 4.2 million veterans have a disability of some type. Not all of those are compensable, but a great many are. At the same time we produced massive numbers of disabled veterans, agencies became more reliant on technology for applicant screening and the amount of good training for HR staff was reduced. Put all of those together, and it became common to see certificates where everyone is a veteran.

People who believe category rating did not work tend to say that because they see hiring managers being unable to hire anyone but veterans for many jobs. They argue that category rating should be done away with.

I think the right answer is to do proper qualifications screening, then apply category rating in such a way that only the very best qualified applicants, with scores that are close to perfect, are in the top category. That still does not address the issue of the large numbers of compensably disabled veterans that show up on certificates, but it ensures that every one who does meets all of the qualification requirements. Whether or not putting every CP or CPS veteran at the top of the list is a good idea is a separate matter that is not related to category rating or the rule of three or any other rating scheme we might devise. It is a consequence of policy decisions and producing large numbers of disabled veterans. Maybe the policy should be changed for veterans of future wars. Changing it for people who have already served and become disabled seems like changing the rules after the fact and I do not see how that could be fair.

So – category rating is not the solution to all problems some early proponents made it out to be. It is also not the root of all evil as some opponents argue. It is a reasonable approach to rating that, if it is applied correctly, can result in hiring managers having larger groups of candidates from which to select. From that perspective, it did what it promised.

This column was originally published on Jeff Neal’s blog, ChiefHRO.com, and was republished here with permission from the author. Visit ChiefHRO.com to read more of Jeff's articles regarding federal human resources matters along with his insights on reforming the HR system.

© 2016 Jeff Neal. All rights reserved. This article may not be reproduced without express written consent from Jeff Neal.

Tags: ,

About the Author (  |   )

Jeff Neal is a senior vice president for ICF International and founder of the blog, ChiefHRO.com. He formerly served in the Obama Administration as Chief Human Capital Officer for the Department of Homeland Security and Chief Human Resources Officer for the Defense Logistics Agency. Jeff is also a Fellow at the National Academy of Public Administration and a Partnership for Public Service SAGE.

Comments

Free Email Updates

Sign up for our free email updates to get our latest news sent directly to your inbox
  • Get our latest news in your inbox each business day
  • More news from your FedSmith.com authors
  • Get the day's closing prices for the TSP funds
  • Get notified via email when new videos are posted