In September, FedSmith posted a performance appraisal survey that garnered over 2,000 responses. The site manager forwarded the results to me for review and comment. In looking through the data from this large sample of Feds, I have found both encouraging data and areas of concern.
Who answered the call?
Before looking at the good, bad and ugly in the responses, some information about the people who responded is in order. We don’t necessarily know which Federal employees read FedSmith, nor is it apparent who chose to complete this survey. Rest assured, the sample is neither random nor validated in any way.
That being said, almost 70% of those responding identify themselves as non-supervisory. Approximately 10% told us they work in human resource/capital management. The remaining respondents identified themselves as supervisory.
I was surprised to see that over a third of respondents say they are evaluated in the context of a pay-for-performance (PFP) system. Slightly over half said they are not. Curiously, about 12% weren’t sure how to answer this question. I have no idea what we can glean from that sizable response. It seems as if most FedSmith readers would know if they live with or without PFP in their work lives. Go figure.
The feedback on feedback
The first question was, “Do you get adequate feedback during the year regarding your job performance?” The decided majority (about 63%) responded in the negative. While not unexpected on my end, it’s clear that our evaluations systems in government are not encouraging the level of interaction agencies might prefer.
Om a recent survey of Federal SESers and GS managers, McKinsey & Company found “…34% of GS12-15 public sector respondents [feel] encouraged to provide honest feedback to people within the agencies (compared with 51% of the SES sample and 48% in the private sector”. In 2000, the Merit Systems Protection Board found that only 20% of their sample responded favorably to the statement, “The performance appraisal rating system has helped increase communications about my job between my supervisor and me.”
Performance appraisals exist (in part) to answer the age-old question, “How am I doing?” While the need for attention and validation may be never satisfied, both studies indicate Federal managers are not adept in this area. As I recommended in a previous article [A Modest Proposal For Improving Federal Supervision], perhaps quarterly feedback should be incorporated into supervisory/managerial appraisal criteria.
The additional two cents
This question also invited written comments. Over 350 respondents chose to add their own statements. Common among them were the following themes:
- No interim reviews (and in some cases, no ratings) occurred during the year;
- Supervisors are too far away to really know what the employee is doing;
- Supervisors lack the background/expertise needed to give meaningful feedback;
- Feedback is vague and of little use to the employee;
- Changes in supervision/leadership have left a vacuum;
- Feedback focuses on negatives rather than future improvements;
- Performance feedback is treated by management as a distasteful chore;
- Feedback is reserved for those who management likes the most;
- Supervisors are too busy with other duties;
Many responses indicated that feedback was done regularly, done well, and management was making a sincere effort. Some respondents complained that such information only came at their request or when reviews were required, however, they did not disparage the quality of the information provided. A few people felt that the ongoing feedback received during the work year made structured conversations less important.
Looking into the mirror
The next question asked, “Do you believe your last performance rating was accurate?” About half of those with an opinion felt their own performance rating was accurate. As one who teaches and writes articles on this subject, I would prefer a much higher number here. By the same token, I see that the responding group was not dominated by malingers and losers. Would a more scientific sample have pushed that percentage higher? Perhaps.
Of interest to me and some reading this summary, about 7% felt unsure as to whether they were accurately rated or not. During my years as a Federal employee, I might have fallen into that category on a few occasions. It begs the question: Are we the best judges of our own performance? [See Let Us Now Praise…Ourselves]
Comments were also invited regarding this question. Among the hundreds received, the following concerns were repeated:
- Pre-determined quotas prevented accurate ratings from being approved;
- Budgets for PFP systems limited the number of high ratings;
- Favoritism among supervisors/managers influenced performance ratings;
- Subjective standards lead to subjective ratings;
- Lack of management familiarity with the job being evaluated;
- Evaluation metrics don’t reflect individual performance/contribution;
- Less visible jobs (“farther from the flagpole”) generally get rated lower;
- Supervisory determinations are commonly overruled in pay pools;
- Higher ratings are limited and rotated among deserving employees.
Hard numbers or soft prose?
In response another question, “How are your performance standards/objectives/expectations generally composed?” almost half of those taking the survey felt their appraisal criteria were purely subjective.
Fewer than 10% view their performance standards as purely “objective”. The remainder felt that their criteria represented a combination of the two. This has implications for those who advocate for GPRA (Government Performance and Results Act) oriented, data-driven standards, SMART objectives, and similar calls for metrics.
Of greater interest to me, was the fourth question which read, “Is your performance actually evaluated against specific expectations?” As many respondents as not felt they were rated by their evaluation criteria. About a fourth of the group wasn’t sure.
When more than a third of people responding feel the written performance appraisal criteria are determinant of their rating, problems are inevitable. (See the United States Code citation below) By the same token, the results indicate that a sizable portion (~37%) felt that standards/objectives really are being used to evaluate their achievement.
Does management manipulate ratings?
Happily, almost two-thirds answered “No” to the question, “Was your last performance rating changed by managers other than your supervisor.” Human resources/capital professional should be concerned about the ~17% who believed such changes did occur to them (and the more than 20% who weren’t sure) as such suspicions can serve to undermine the credibility of appraisals. Overall, however, I was pleased that the first-level supervisor’s assessment is accepted for the much larger group.
Perhaps the worst news came in response to question 6: Do you believe your agency has a “quota system” for performance ratings?” Readers were given the choices of “Yes”, “No”, and “Unsure”. Over 62% responded “Yes” and 19% more were “Unsure”. That left less than 19% feeling as if some desired rating curve or profile wasn’t at work.
As law and Office of Personnel Management (OPM) regulations expressly prohibit any “forced distribution” of ratings, this survey raises serious concerns. We in the HR/HC community have been hearing more about “bell curves” as PFP has proliferated. It seems as if government needs to admit that employees are rated “on a curve” or act to stop the practice.
Perceptions of bias
Another statistic that should raise some level of alarm is contained in responses to question 7: “Was your last performance rating affected by supervisory favoritism?” While 42% simply said “No”, an equivalent number answered “Yes”.
This is an area that OPM should examine before promoting further implementation of pay-for-performance. Despite any concerns regarding the reliability of the sample, these numbers reflect badly on Federal leadership in the trenches. My own experience with management was much more favorable, but that was me… and decades ago.
I don’t know what comparable private sector data might reveal, but the impression from this questionnaire isn’t good. Federal law states in part,
Under regulations which the Office of Personnel Management shall prescribe, each performance appraisal system shall provide for establishing performance standards which will, to the maximum extent feasible, permit the accurate evaluation of job performance on the basis of objective criteria (which may include the extent of courtesy demonstrated to the public) related to the job in question for each employee or position under the system; and evaluating each employee during the appraisal period on such standards… (5 USC 4302)
The results contained in the FedSmith survey provide decisive findings that this isn’t the case. Most comments (from previous questions) point toward generic favoritism as the larger problem; however, several comments cited the perception of EEO bias as well.
Where’s the data?
Question 8 asked, “Does your supervisor have a system of records or notes concerning your individual performance?” Twenty-five percent responded “Yes”. That left three quarters of the two thousand respondents either unsure or answering “No”.
As was noted in one of my past articles [Appraisals, Objectivity, and the Little Black Book], it’s tough to do a meaningful performance appraisal in the absence of written data. I don’t mean to imply that data must be quantitative, but anecdotal recall at the end of the year is often a ticket to misunderstandings, if not inaccuracies.
Toss this one out
The final question was invalidated by the number of people who answered it. It asked, “If you are evaluated under a pay-for-performance or pay banding system, do you prefer pay banding to grades and steps?”
By my reckoning, only about 800 respondents said they work in a PFP environment, yet more than double that number responded to the question. While well over half said “No” and over a fourth answered “Not Sure”, I cannot determine what numbers are valid and therefore reserve comment. Other studies can provide readers with more conclusive data.
Looking ahead
Many thanks to the many FedSmith readers who took the time to answer these questions and provide the myriad written comments received.
Your answers to the survey should point the way toward much-needed research and corrections when it comes to annual performance evaluations. Moreover, these findings (as unscientific as they may be) raise serious concerns when viewed by a human resources/human capital community that advocates for proliferation of pay-for-performance.
The Office of Management and Budget’s (OMB) Chief Performance Officer recently testified before a Senate subcommittee that his agency is designing a new performance management system. He would be well-advised to consider the supervisory/management infrastructure needed to implement his agency’s GPRA-focused plans.
Moreover, the current staffs at OPM and OMB should remember that objectivity was demanded by Jimmy Carter’s Civil Service Reform Act of 1978. Such government-wide requirements went nowhere in an environment where individual performance data is often not maintained by real supervisors. “Because we say so” is unlikely to change appraisal practices some of us have witnessed for decades.
