Category Rating – Does It Work?

By on July 10, 2016 in Human Resources with 15 Comments

Image of 3d person standing next to category rating scale

In my most recent blog post I said part of the current difficulties agencies are experiencing with hiring non-veterans is caused by how they have implemented category rating and not by veteran preference itself. I promised to address category rating in a future post. This is that (long) post.

When I talk to federal employees, hiring managers, and veterans about federal hiring practices, it is common for them to have little understanding of category rating and how it works. Unfortunately, I have also found some HR folks who do not understand how it is supposed to work. So – I am going to try to demystify it a bit and show how it is supposed to work and how it sometimes works badly.

To understand category rating we need to understand the step-by-step process for reviewing applications and then go back in time and understand category rating’s precursor, the “rule of three.” For many years the rule of three was gospel when it came to federal hiring. Agencies rated applicants and assigned a numerical score on a hundred point scale. Really, however, the scale was only 30 points, because any qualified applicant got a score of 70. So the basic score range was 70 – 100. But this is the government, so simple is simply not good enough.

The Process

Once an agency has applications for a job, it has to decide who to refer on a certificate (the document listing the candidates a selecting official may choose from). That begins with determining basic qualifications. Every job has qualifications requirements that are laid out in OPM qualification standards. They are usually written in the form of a number of years of experience, education that is required for the job, and combinations of experience and education that would qualify an applicant. You find them in the beginning of most applicant questionnaires in the form of a question or two. An applicant who does not meet the basic qualifications requirements is determined to be not qualifiedand is out. Scores and veteran preference are not a factor at this stage of the process. It is important to note that HR and subject-matter experts (SMEs) are supposed to play a big role in this process. Regardless of how an applicant answers the qualifications questions, HR (with advice from SMEs) is supposed to review the resume and make a qualifications determination. For example, if an applicant says “Yes” to the question regarding specialized experience, but has no such experience identified in his/her resume, HR is supposed to override the answer and rate the applicant as not qualified. The systems HR folks use to process applications, such as Monster, Acendre or USA Staffing, are all designed to allow and document such changes.

Some jobs have “Selective Factors” –  qualifications that are also important for the job and are already required when the person starts the job. They typically require extensive training or experience to develop, are essential for successful performance on the job, and cannot be learned on the job in a “reasonable” amount of time. An applicant who does not meet a selective factor is ineligible and is screened out. That is why selective factors are sometimes referred to as “screen out elements.” As with qualifications requirements, HR is supposed to verify that the application reflects the required experience and override it if the applicant claims experience that the resume doe not reflect.

Based upon conversations with a lot of hiring managers in different agencies, I believe the number one problem agencies are experiencing now is that a thorough review of applications by humans is just not happening. HR folks are erring on the side of passing through to the next phase everyone who says s/he is qualified. Some HR folks tell me they do not have the time to do good reviews. Others say the rating process will wash out any unqualified candidates. I have even had HR folks tell me they believe it is illegal for HR to override what an applicant says on a job application. As you will see below, failure to do proper qualifications screening is a problem.

Once HR has completed screening applicants to ensure they are qualified, the process moves on to the actual ratings. The old process of the rule of three worked this way. Based on applicant responses to questions or reviewing their applications, HR assigned a score. That score has additional points added for veterans. Here are the categories of veterans and the extra points they earn:

  • CPS: 10-Point 30 Percent Compensable Disability Preference based on a service-connected disability of 30% or more
  • CP: 10-Point Compensable Disability Preference based on a service-connected disability of 10% or more, but less than 30%
  • XP: 10-Point Disability Preference; granted to recipients of the Purple Heart, persons with a non-compensable service-connected disability (less than 10%)
  • XP: 10-Point Derived Preference granted to widow/widower or mother of a deceased veteran, or spouse or mother of a disabled veteran;
  • TP: 5-point preference

Veteran preference points are added to scores, then they are put in order to determine who would be referred to the selecting official. This is where it gets a little complicated. The list on the left is a sample list of applicants and their raw scores + preference points. The list on the right is the same applicants with the scores put in the order they would go on a referral list.

Image of charts showing veterans preference scores

 

With the rule of three, the hiring manager would get a list that includes the top 3 candidates on the right. Jon Snow is well qualified, with a basic score of 88 points and an additional 10 for being a compensably disabled veteran (below 30%). Fred Smith, the next candidate on the list, meets the basic qualifications but no more. He is near the top of the list because he is a 30% disabled veteran. The most qualified candidate in this example is Jim Eliot. He has a perfect score. In this example, the hiring manager will get a list that includes the top 3, but s/he cannot select Jim Eliot because doing so would require passing over a veteran (Jon or Fred).

There are many more complexities in the rule of three, such as the three passovers and you are out rule when multiple jobs are being filled from the same list, but getting into those would take far too much time. If you are interested in knowing about that one, go to “Auditing a Certificate of Eligibles Under the Traditional “Rule of Three” Procedures” in OPM’s Delegating Examining Operations Handbook. Suffice it to say that the rule of three was hard to administer and drove HR, hiring managers and applicants crazy. Category rating was intended to make it much less complex.

Category Rating

Category rating was authorized by the Chief Human Capital Officers Act of 2002. It says “The Office [of Personnel Management] …may establish category rating systems for evaluating applicants for positions in the competitive service, under 2 or more quality categories based on merit consistent with regulations prescribed by the Office of Personnel Management, rather than assigned individual numerical ratings. Within each quality category established under subsection (a), preference-eligibles shall be listed ahead of individuals who are not preference eligibles. For other than scientific and professional positions at GS–9 of the General Schedule (equivalent or higher), qualified preference-eligibles who have a compensable service-connected disability of 10 percent or more shall be listed in the highest quality category.”

Category rating was an optional procedure that many agencies chose not use, until President Obama’s 2010 hiring reform mandated its use instead of the rule of three. OPM’s guidance on category rating gives agencies a lot of flexibility in how they implement it. OPM also published a briefing and a set of Q&A that explain the process in more detail.

Agencies are required to have at least 2 categories, although many use 3 or 4. Applicants are put into categories and then listed by category in preference order. Although the statutory requirements for category rating do not include assigning numerical scores, many agencies use scores because they are easier to administer. Examples of some agency policies are herehere and here. When agencies assign numerical scores, extra points are not added for veterans.

Image of chart showing example of veterans preference category ratingThe sample list of applicants from above looks very different in a representative category rating scheme. Where before we had 3 candidates on the list, but only 2 who could be selected (Jon Snow and Fred Smith), now we have 6 in the top category, only 4 of whom could be selected. Even though Jim and Betty have high scores, they cannot be selected because doing so would require passing over qualified veterans. From the perspective of a hiring manager, the result could be either bad or good. Bad because s/he still cannot select the non-vet with a perfect score. Good because 2 veterans with high scores who were not within reach before are now avail be to be selected. From the perspective of the veterans on the list, it can also be bad or good. Bad if you are Jon or Fred, because now there are 2 better qualified veterans who are more likely to be selected. Good from the perspective of George and Steve, because now they are within reach and can be selected. It is still bad for Jim, because there is no way he is going to get this job, regardless of how the rating process is done. That is not a change – he would not get the job under the rule of three and he will not get it under category rating.

Is Category Rating an Improvement?

Category rating was supposed to eliminate the confusion caused by the rule of three and provide agencies with a more flexible rating process that gave hiring managers more candidates to consider. For the most part, it does that. Some people believed category rating would reduce the impact of veteran preference. It did not do that. In fact, unless the top category is very narrow (e.g., 98 to 100 points), the impact of veteran preference was probably amplified. When agencies use broad ranges for the top category (e.g., 90 – 100 points), hiring managers are seeing more candidates, but all of them are veterans. When they fail to do good qualifications screening, the result can be that unqualified applicants who are compensably disabled veterans get put into the rating process when they should have been screened out. Then they float to the top, and the process breaks down.

When category rating was implemented, most people did not know we would be in 2 wars for many years, and that those wars would produce a high number of disabled veterans (1.2 million, or 1 in 3, veterans of Iraq and Afghanistan are disabled in some way). A total of 4.2 million veterans have a disability of some type. Not all of those are compensable, but a great many are. At the same time we produced massive numbers of disabled veterans, agencies became more reliant on technology for applicant screening and the amount of good training for HR staff was reduced. Put all of those together, and it became common to see certificates where everyone is a veteran.

People who believe category rating did not work tend to say that because they see hiring managers being unable to hire anyone but veterans for many jobs. They argue that category rating should be done away with.

I think the right answer is to do proper qualifications screening, then apply category rating in such a way that only the very best qualified applicants, with scores that are close to perfect, are in the top category. That still does not address the issue of the large numbers of compensably disabled veterans that show up on certificates, but it ensures that every one who does meets all of the qualification requirements. Whether or not putting every CP or CPS veteran at the top of the list is a good idea is a separate matter that is not related to category rating or the rule of three or any other rating scheme we might devise. It is a consequence of policy decisions and producing large numbers of disabled veterans. Maybe the policy should be changed for veterans of future wars. Changing it for people who have already served and become disabled seems like changing the rules after the fact and I do not see how that could be fair.

So – category rating is not the solution to all problems some early proponents made it out to be. It is also not the root of all evil as some opponents argue. It is a reasonable approach to rating that, if it is applied correctly, can result in hiring managers having larger groups of candidates from which to select. From that perspective, it did what it promised.

This column was originally published on Jeff Neal's blog, ChiefHRO.com, and has been reposted here with permission from the author. Visit ChiefHRO.com to read more of Jeff's articles regarding federal human resources and other current events along with his insights on reforming the HR system.

© 2016 Jeff Neal. All rights reserved. This article may not be reproduced without express written consent from Jeff Neal.

Tags:

About the Author

Jeff Neal is a senior vice president for ICF International and founder of the blog, ChiefHRO.com. He formerly served in the Obama Administration as Chief Human Capital Officer for the Department of Homeland Security and Chief Human Resources Officer for the Defense Logistics Agency. Jeff is also a Fellow at the National Academy of Public Administration and a Partnership for Public Service SAGE.

Post a Reply

Your email address will not be published. Required fields are marked *

15 Replies

Comments RSS

  1. Chasseur1814 says:

    To be honest, I dismiss as nonsensical any commentary that contains sentences such as this one from the article above’ “Once an agency has applications for a job, it has to decide who to refer on a certificate.” “Agencies” never make decisions, nor do “bureaus”, “administrations”, or the ubiquitous “government”. The only entity that makes decisions are people. Stop reifying everything in government, and start holding individuals accountable. The first step in the hiring process should not be a clerk, without even a university degree deciding (based on their low level cognitive abilities and zero management experience) who meets the selection criteria. Until, local manager are given the authority and wherewithal to make hiring decisions based on their best judgment as managers then the federal hiring process will continue be nothing more and endless morass and quagmire. Oh, while at, stop calling it “Human Capital” there is nothing human about the way those offices are run, and no one in Human Capital even posses a rudimentary understanding of the principle capital formation or conservation.

  2. US Army Worker says:

    “It is still bad for Jim, because there is no way he is going to get this job, regardless of how the rating process is done.” Actually this is not true. The four veterans in the “Best Qualified” group could all decline the position, and then Jim could be selected, and accept the position. This happens all the time when you have applicants who apply for every open position out there. I’ve actually seen lists within my agency that contain anywhere from 5-10 applicants or more, and all applicants decline because I’m assuming they check the location of the job AFTER the offer is made. We have employees in very remote areas, some of which make jobs hard to fill.

    • The Insidiator says:

      Isn’t selecting a location where you are willing to work part of the application process? It seems like a terrible waste of HR time to apply for a vacancy when you know you are not willing to work at the duty station. Are the positions you’ve seen where so many people decline offers those jobs advertised at 15 different offices and the candidate happens to be offered the position on the one place they don’t like? Or perhaps do people want interview practice?

      • US Army Worker says:

        Yes, it IS a waste of HR’s time…believe me! It seems like from what we see, the applicants are just applying for everything under the sun that they think they may qualify for, sometimes vacancies open only at one location, and someone vacancies with multiple duty locations. Then during the interview process when asked the question “Would you accept this position if offered?”, they reply “Yes.” Then you make the formal offer, and they decline the position, so onto selectee #2, and #3, and so on.
        With the jobs that are advertised at multiple locations, the applicant has the ability to only apply for those duty locations that are of interest to him/her.
        Another problem is the length of time it’s taking to get jobs filled. We’ll have positions that are advertised in March for example, not get a referral list until sometime in May, maybe even June, and not make an offer on the position until July or August. I can’t blame applicants for turning jobs down in these types of situations…they’ve probably already found other employment at that point.
        The problem isn’t just any one thing, it’s a myriad of issues causing the bigger issue of getting jobs filled.

  3. Michael D Robbins says:

    I don’t like how some articles on Fedsmith, this one in particular, seem to paint HR employees with the same brush (incompetent, uncaring and slow). I don’t like the section where the author points to how HR is “supposed” to do certain things. HR has become extremely overworked but I don’t see HR randomly disregarding their responsibilities.

  4. The Insidiator says:

    Ugh. Thank you for what I’m guessing is the most concise description possible of this process. It still irks me that these policies are written such that the most qualified applicant (based on real-world experience and training; the Jim Eliots of the world) might not even get an interview.

    There must be a way to address the spirit of preference eligibility while not completely blocking perfect, non-preference, external candidates. Number system and tie-goes-to-the-preference-eligible? More selective factors so that subject experts are easier to bring in (though somehow controlling for abuse)?

    • msgrowan says:

      Selective factors are used to identify essential KSAs that are typically unique to a specific position apart from the occupation’s general qualification requirements (e.g., working knowledge of Spanish in a SSA contact representative position located in an area where a large Spanish-only speaking population is to be found). These factors cannot be ginned up willy-nilly but must arise out of an intensive job analysis process used to identify them in the first place prior to any subsequent recruitment action. If an agency proffered a claim to have identified a newly identified selective factor AFTER undertaking recruitment action for a given position, it would – or should – come under intense suspicion of seeking to skew the applicant pool inappropriately, as the “abuse” issue is not as “controllable” as you seem to presume.

  5. irdeggman says:

    It should be noted that veteran’s preference only applies to non-merit promotions, i.e., jobs that announced for multiple areas of consideration (that is to say outside of an single command). So the above listed on order of referral lists are only for positions outside of the merit promotion category.

    • NM Cankaya says:

      Veterans’ preference only applies when a job is open to the public, i.e. US Citizens. It is not a factor in any merit promotion announcements, i.e. open to a specific agency or government worldwide. There are special hiring authorities and legislation that lets us consider veterans under merit promotion in some circumstances (namely VEOA, VRA, 30% disabled veterans), but that is not preference. Our agency asks that managers and leaders focus on history and outreach response to determine how to announce, internal to the agency, internal government-wide, or open to the public. We almost never do two announcements to fill a job – it is a waste of our time and the applicants. Our internal merit promotion rate is fairly high.

      • exceeds performance says:

        Sounds like a lot of inbreeding and not good for the taxpayers

        • Paul in cAshburn says:

          All gave some. Some gave all.
          Veteran preference is the right thing to do – and George Washington instituted that principle.
          How things get implemented… Well, that’s done by imperfect humans – and we all fall into that category.

      • William Dieterich says:

        Under the current law the veterans preference applies at all times https://chiefhro.com/2016/06/2… If the upcoming defense bill passes as is then it will switch back to veteran preference only applying to to get you hired after that it will be up to your skills and ability.

        • NM Cankaya says:

          “Preference applies in hiring from civil service examinations conducted by the Office of Personnel Management (OPM) and agencies under delegated examining authority, for most excepted service jobs including Veterans Recruitment Appointments (VRA), and when agencies make temporary, term, and overseas limited appointments. Veterans’ preference does not apply to promotion, reassignment, change to lower grade, transfer or reinstatement.”
          From OPM’s Vetguide. The last sentence refers to merit promotion – essentially most of the announcements you see on USA Jobs, that are limited to government or agency employees. It is a common mistake to mistake merit promotion with new appointments, i.e. “hiring”. Promotions, transfers, changes to lover grade, and reinstatement are merit actions exempt from VP and are not considered “hiring”. If your HR office is extending it to merit promotion they are violating CFR. They might be advertising jobs several ways, and selecting off DEU type certs (open to the public) instead of merit promotion certs, in that case, they are probably doing it right.

          • irdeggman says:

            Correct and to fine tune it . . here is an e-mail chain I received from regional HR (almost all are regional now as a “cost-savings” measure).:

            Mr. XXXXX,

            If this position had been announced internally as Merit
            Promotion then veteran preference would not apply. However, as previously stated this was
            announced by our external recruitment division which has delegated examining authority
            from OPM to recruit and hire for competitive and excepted service positions.

            It does not matter if this position is filled internally
            or externally, applicants for GS positions must still meet OPM qualifications
            for one year of specialized experience at or equivalent to the GS-13 grade
            level or pay band in the federal service or “equivalent experience in the
            private or public sector.”

            OCHR XXXX

            —–Original Message—–

            From: XXXX

            To: OCHR-XXXX

            This is what it specifically states:

            ” When Preference Applies

            Preference in hiring applies to permanent and temporary
            positions in the competitive and excepted services of the executive branch.
            Preference does not apply to positions in the Senior Executive Service or to
            executive branch positions for which Senate confirmation is required. . .”

            And then it goes on:

            “. . . . Veterans’ preference does not apply to
            promotion, reassignment, change to lower grade, transfer or
            reinstatement.”

            Again this is specifically from the OPM site (referenced
            in previous e-mail) under veteran’s services…veteran’s preference in
            appointments.

            So is this considered a promotion or not?

            Since all GS promotions (except for those in a recognized
            training program) are “competitive” and this was classified as such.
            . . .it should be classified a promotion and veteran’s preference does not
            apply.

            My objections still stand. . .

            —–Original Message—–

            From: OCHR-XXXX

            To: XXXX

            This announcement was open to any U.S. citizen through
            our Delegated Examining Unit which has Delegated Examining Authority.

            Preference applies in hiring from civil service
            examinations conducted by the Office of Personnel Management (OPM) and agencies
            under delegated examining authority, for most excepted service jobs including Veterans
            Recruitment Appointments (VRA), and when agencies make temporary, term, and
            overseas limited appointments.

            OCHR XXX

            —–Original Message—–

            From: XXXX

            To: OCHR-XXXX

            Hmmm. . . . but from the OPM site itself.

            Specifically it does not apply to promotions.

            The position in question had the following prerequisite:

            “Your résumé must demonstrate at least one year of
            specialized experience at or equivalent to the GS-13 grade level or pay band in
            the federal service or equivalent experience in the private or public
            sector.” Hence this is considered
            a “promotion” since there is a prerequisite of lower grade
            experience..

            http://www.opm.gov/policy-data

            “When Preference Applies

            Preference in hiring applies to permanent and temporary
            positions in the competitive and excepted services of the executive branch. Preference
            does not apply to positions in the Senior Executive Service or to executive
            branch positions for which Senate confirmation is required. The legislative and
            judicial branches of the Federal Government also are exempt from the Veterans’
            Preference Act unless the positions are in the competitive service (Government
            Printing Office, for example) or have been made subject to the Act by another
            law.

            Preference applies in hiring from civil service
            examinations conducted by the Office of Personnel Management (OPM) and agencies
            under delegated examining authority, for most excepted service jobs including
            Veterans Recruitment Appointments (VRA), and when agencies make temporary,
            term, and overseas limited appointments. Veterans’ preference does not apply to
            promotion, reassignment, change to lower grade, transfer or reinstatement.

            Veterans’ preference does not require an agency to use
            any particular appointment process. Agencies have broad authority under law to
            hire from any appropriate source of eligibles including special appointing
            authorities. An agency may consider candidates already in the civil service
            from an agency-developed merit promotion list or it may reassign a current
            employee, transfer an employee from another agency, or reinstate a former
            Federal employee. In addition, agencies are required to give priority to
            displaced employees before using civil service examinations and similar hiring
            methods.

            Civil service examination: 5 U.S.C. 3304-3330, 5 CFR Part
            332, OPM Delegation Agreements with individual agencies, OPM Examining
            Handbook, OPM Delegated Examining Operations Handbook; Excepted service
            appointments, including VRA’s: 5 U.S.C. 3320; 5 CFR Part 302; Temporary and
            term employment: 5 CFR Parts 316 and 333; Overseas limited employment: 5 CFR
            Part 301; Career Transition Program: 5 CFR Part 330, Subparts F and G.”

            So my protest seems to stand based on OPM site
            information itself.

            —–Original Message—–

            From: OCHR-XXX

            To: XXXX

            By law (Title 5 USC, Section 2108), veterans who are
            disabled or who serve on active duty in the Armed Forces during certain
            specified time periods or in military campaigns are entitled to preference over
            non-veterans both in Federal hiring practices and in retention during
            reductions in force (RIF).

            OCHR Norfolk Operations Center

            —–Original Message—–

            From:XXXXX

            To: OCHR-XXXX

            I do not believe there is a “law” that drives
            this.

            There was a Presidential “suggestion” and an
            OPM guidance.

            Those are not “laws”.

            —–Original Message—–

            From: OCHRXXX

            Mr. XXXX,

            The only applicants that were referred were the highest
            qualified applicants with veterans preference.
            By law veteran’s have preference over non veterans and if we have a
            sufficient number of qualified applicants with veterans preference, they will
            receive first consideration. Therefore, only applicants with veterans preference
            were referred for this position. The
            notification letter does not allow us to indicate that on the letter, so it
            just states that you were not among the highest qualified.

            We appreciate your continued interest in Federal
            employment with the Department of the Navy.

            OCHR XXXX

            ___________

            Who is the recruiter for this position?

            I wish to know why my resume was not forwarded – i.e.,
            why my rating wasn’t higher.

            Thank you in advance for answering me.

    • Michael D Robbins says:

      Veteran’s preference applies to merit promotion announcements if a manager is selecting a VRA applicant with no other status. If multiple VRA eligible candidates are referred, a thirty percent disabled (CPS) vet should be selected over a TP vet. This is according to the vet guide on the OPM website.

Top