WHAT IS A VALIDATED 360 FEEDBACK SURVEY?
Validation is a research process that aims to show whether an instrument actually measures what it was designed to measure. It is a valuable feature in any published feedback survey. There are many types of validation studies. In the case of 360 degree feedback for training and development purposes, survey designers strive for construct validity. Construct validity is evaluated by determining how well the concepts behind the instrument account for an individual's performance on the instrument. For example, the Survey of Management Practices from the Clark Wilson Group's Task Cycle series measures the universal behaviors that are predictive of an effective manager. A manager with a strong record of success should have high scores on the survey whereas a manager whose performance has been weak should have low scores.
360 Survey FAQ’s
What is a 360 Survey?
- It is a type of organizational survey that links an individual’s skills to job effectiveness. Also called multi-rater feedback, it typically uses a survey instrument with 40-100 statements about business skills and behaviors specific to an organizational role, such as manager. The feedback recipient rates him or herself on those statements, as do a group of feedback-givers. Feedback providers may include boss, subordinates, peers, team members, students, customers, and even family members.
- Feedback surveys can help anyone become more confident of their strengths and more aware of their challenges. When combined with training and development programs, 360 feedback can dramatically increase individual job effectiveness quickly and cost-effectively.
What are the advantages of using a 360 Feedback Survey?
- Accuracy (much more accurate than performance appraisals)
- Richer understanding of performance (you get multiple perspective on the individual’s behavior)
- Promotes open communication
- Reinforces organizational values
- Easy and fast to implement
Can 360 feedback be used for job performance appraisal?
- We strongly recommended that 360 feedback be used only for developmental purposes and that the actual scores remain confidential to the participant. This recommendation precludes use of 360 feedback for performance appraisal. The only exception is the case of succession planning projects, where sharing of data is voluntary. In these situations, the participants know in advance their results will be reviewed by a combination of human resource specialists, upper management and their own managers.
- The pressure to use multi-rater feedback for various performance management applications is partly due to the poor quality and inconsistent execution of conventional appraisal systems. Yet even in organizations where these systems are administered well, there are still those who recommend that multi-rater feedback surveys be used for appraisal. Management and human resource professionals, impressed with the strong psychometric standards of surveys such as the Task Cycle instruments, want to bring that precision into the appraisal process. Another set of reasons has to do with accountability. It is sometimes thought that if the 360 feedback results are more public, or at least shared with the person’s manager, the participant will be forced to improve performance.
- Those of us on the side of “development-only” warn of all sorts of dangers inherent in changing the privacy rule and adding appraisal to the list of applications. For one, there is the danger that employees will “game the system”. This not only destroys the appraisal process, it also impacts the norms. “Grade inflation” is one consequence. Yet even more serious is the loss of the usefulness of multi-rater feedback surveys. When the scores become meaningless, so does the feedback.
How Can We Increase Job Competency with 360 Feedback?
- Multi-rater feedback excels at identifying perceptions of behaviors — both the individual’s self-perceptions and the perceptions of others–as they relate to role-specific core competencies. Once soft spots and blind spots are identified, training and development can be offered. By the same token, unrealized strengths may be identified and further developed.
- Many organizations have conducted extensive studies in order to article their important, role-specific core competencies. A core competency statement describes the capability required to do a job successfully. Performance Programs is often asked to use those studies to create customized feedback questionnaires. We review each of the competencies for its potential “observability,” seeking the behavioral aspects of each competency. We can then create customized 360 feedback questions that focus on observations of behaviors. Unlike personality, behavior can be changed through training and coaching. The customized 360 feedback questionnaire is often used in conjunction with training and development programs.
- Many core competencies are very similar across organizations and have been studied elsewhere. The Task Cycle® Surveys of the Clark Wilson Group were developed to identify universal core competencies in management, leadership and team roles. They have been validated and shown to predict effectiveness in the roles they measure.
- Whether an organization uses custom-developed 360 or validated 360 such as the Wilson surveys, the combination of competency statements, a 360 questionnaire, and training is a very powerful way to help individuals increase the competencies that matter most.
Are there multiple approaches to 360 feedback?
There are 3 basic “organizing principles” to consider:
- You can use validated surveys, which use research linking specific skill areas to actual work outcomes. These offer the power of predictability — if the participant exhibits a certain set of linked behaviors, he or she will be more effective.
- You can use competency studies, based on internal analysis of required skills and abilities. Custom questionnaires are developed to measure internal progress against internal goals and needs.
- You can use vision and/or value statements, based on the need to move an organization in a direction. These are extremely useful feedback to management about the alignment between mission and actual behaviors, helping them fine tune their leadership activities.
In all cases, well-designed 360 feedback focuses on observable behaviors.
What is the value of combining 360 feedback with personality assessment?
The combination of 360 feedback with personality assessment can provide an individual with self-understanding that would be difficult to achieve in other ways. It also helps a coach gain deeper insight for assessment and development purposes.
Personality testing can show a person’s potential “fit” with a job or an organization and what motivates them to succeed. Validated 360 feedback shows what a person does and how their behaviors impact others. The combination of these perspectives offers these benefits:
- Individuals better understand their own personality and motivations.
- They understand how their skills are perceived by others.
- Coaches are equipped with well rounded material for feedback sessions.
- Reporting and interpretation can be unified.
- This combined assessment supports internal promotion decisions or can be used for training and development.
Research shows that realistic self-awareness is a major asset in career development — and helpful in the prevention of career derailment. A combination of behavioral and personality feedback helps people understand why a particular tendency or uniqueness may be an advantage in one setting, but a disadvantage in another. They get past the idea that they need to “give up” some behavior or that their personalities are flawed. Instead, the assessment helps the participant to be themselves and at the same time be more effective in a business setting.
What is a validated 360 feedback survey?
Validation is a research process that aims to show whether an instrument actually measures what it was designed to measure. It is a valuable feature in any published feedback survey. There are many types of validation studies. In the case of 360 degree feedback for training and development purposes, survey designers strive for construct validity.
Construct validity is evaluated by determining how well the concepts behind the instrument account for an individual’s performance on the instrument. For example, the Survey of Management Practices from the Clark Wilson Group’s Task Cycle series measures the universal behaviors that are predictive of an effective manager. A manager with a strong record of success should have high scores on the survey whereas a manager whose performance has been weak should have low scores.
Validated 360 surveys meet the following criteria:
- The behaviors are observable.
- The behaviors can be trained.
- The behaviors do not overlap.
- They are based on skills that research has shown are correlated with actual job performance.
Each of the following techniques may be employed in creating validated 360 feedback:
- Judgmental and logical analysis
- Correlational analysis
- Process analysis
- Analysis of group differences
- Changes over timeResponsiveness of scores to experimental treatment.
How are custom 360 feedback questionnaires created?
Core competencies are translated into one or more observable behaviors. The survey designer sorts these behaviors into two categories: ones that can be trained and ones that can’t. Trainable behaviors are good candidates for the survey. If your survey is intended for development, we use only questions relating to these behaviors. Behaviors and characteristics that aren’t likely to be affected by training are typically based on personality, temperament and values. These are generally not good candidates for developmental 360 feedback, but they may have a place in other types of assessments.
If you wish to measure the effectiveness of vision or value statements, they also need to be assessed by questions that reflect a behavior. High scores reflect behavior consistent with the value, low scores reflect inconsistency. Group composites, showing how well groups of employees actually “live the values” are very useful springboards for training and executive discussion.
Customized 360 feedback is usually not validated. However, an organization can obtain validated items or groups of items to create a hybrid survey. For validated competencies, the Task Cycle® Surveys from The Clark Wilson Group are an excellent source. If your organization’s competencies match part or all of these, you can capitalize on this research by adding them to your custom 360 feedback questionnaire.
Also, you may wish to access questionnaire items that have norms for a similar employee population. Many core competencies are very similar across organizations and have been studied elsewhere. You may be able to obtain scales that have been validated on published surveys from Clark Wilson Group.
Can we promote mission and values alignment through 360 feedback?
Organizations can custom-design a 360 feedback survey around mission and values statements. The objective of these surveys is to ensure that the stated values are actually being embraced and practiced by all levels of an organization and becoming part of the organization culture. Two survey approaches can be used for this endeavor:
- Values-based 360 feedback for managers and executives is a good choice when you suspect that individual managers are blocking progress. Key issues of fairness, diversity, ethics, innovation, and other values-driven topics can be addressed. Vision and value statements in 360 feedback can determine the alignment between the stated value and the actual behaviors of employees. These questionnaires are typically structured so that high scores reflect behavior consistent with the stated mission or values, and low scores reflect inconsistency. Group composites, showing how well groups of employees actually “live the values,” are very useful springboards for training and executive discussion. If you choose 360 feedback for measurement of values, it will contain questionnaire items that are observable and can be influenced by training and development. A sample statement might be: “The management of this organization encourages innovative thinking.”
- A general employee survey is a good choice for researching the extent to which employees can observe the mission, vision, and values in actual daily practice at all levels. Do they feel the stated values and vision are part of the company culture?
Where does 360 feedback fit in the world of human resources assessments?
Most individual assessments used in business fit four categories. All are discussed below. Performance Programs offers products in all categories.
1) Assessments of process-based competencies
Most published or customized 360 feedback surveys are assessments of process-based competencies. These surveys are usually designed to illuminate areas that people can change through training and development and they avoid issues of personality, temperament or values.
Typical items on a 360 feedback survey might include:
- “This individual tries new things for improvement.” (rate from “never” to “always”)
- “This individual expresses appreciation for people’s efforts.” (rate from “never” to “always”)
- “This individual adapts readily to new team members.” (rate from “never” to “always”)
- The point of 360 feedback is for the individual to become aware of competencies and soft spots (or blind spots) so they can develop appropriate skills and behaviors for greater effectiveness in a role. Designers of these instruments strive to identify skills that distinguish between high and low performers in designated roles. Clark Wilson Group’s Task Cycle Surveys are an excellent example. History of the Task Cycle Theory, first of their kind.
2) Personality Tests & Inventories
The name “personality test” is used loosely, covering everything from fun questionnaires on the Internet to serious tests that detect mental pathology or, in the case of job personality tests, how well suited a person is to a particular job. So, it is important to make some distinctions.
Many famous personality tests, such as Minnesota Multiphasic Personality Inventory (MMPI) and the California Psychological Inventory (CPI) were designed for a clinical context. Clinical instruments usually are not appropriate for career counseling or development because they detect psychopathology, and many ask questions that are not appropriate to a business situation. (However, clinical tests are used when the applicant’s state of mind is important. Pilots, military Special Forces, law enforcement officers, or nuclear power plant operators are examples.)
The second type of personality test looks at how we believe we are known by others based on our observable behaviors. These instruments ask respondents to check off or rate items that best describe themselves as they would react under various circumstances.
Sample statements might include these:
- “My success depends on how others see me.”
- “I am careful to consider the other person’s point of view.”
- “Many of the managers I’ve worked for were incompetent.”
- “I like being in front of a group.”
- For training purposes, self-assessments do not give the information needed to guide changes in behavior. This is why it is often useful to combine personality assessment with 360 feedback.
- Performance Programs offers an array of personality reports based on the Hogan Personality Inventory and Hogan Development Survey.
3) Measurements of styles and attributes
“Type” or style indicators are, from an assessment perspective, hybrids. They measure personality tendencies but also reflect learning history. The well-known Myers-Briggs Type Indicator fits in this category.
These assessments reveal general characteristics or attributes, and patterns of personality traits. They result in such statements as “John is a participator [or a driver, or an analyst, or a pacesetter].” These give a basis for understanding the dynamics of one’s interactions.
Style assessments generally do not assess role-specific skills or behaviors. They can measure how we change our style of interaction over time, however. While personality tendencies remain essentially the same, style can shift as we learn from experience.
4) Motives, Values and Interest Questionnaires
Motives, values and interest questionnaires probe how you would like your work life to be and clarify career direction. Many people confuse what they want to do with what they think they should do.
Because actual job performance relies on motives and values, questionnaires of this type can be very useful. These typically ask respondents to rate the importance of job security, salary, the opportunity to display creativity on the job, and many other job-related preferences. They can help clarify, for instance, whether you prefer people contact more than most people, prefer to be in charge or in a supporting role, or whether you are more independent than other people in your field.
Some examples of statements on these questionnaires include the following:
- “I’d like to be famous.”
- “I need to work by myself.”
- “Show me the money.”
- “I love to run things.”
- “I can’t stand working in the same place every day.
Performance Programs offers reports based on the validated Motives, Values, Preferences Inventory from Hogan Assessments.
What are the major planning considerations for 360 feedback?
Why do you want to use 360 degree feedback surveys? Some common reasons include:
- Executive assessment and development
- Succession planning
- Core competency assessment for specific roles
- Skill development for new supervisors, managers, team leaders and executives
- Team development
- Consultative and service skills development
What outcome are you hoping to achieve with 360? Some common goals include:
- Enhanced leadership & vision
- Organizational bench strength
- Adaptability to rapid change
- Up-leveling individuals to new roles
- Increased team commitment
Have you conducted a 360 survey before? Administrative considerations:
- How many different locations?
- How many different languages?
- Do you have a preference for how the survey will be administered?
- Mailed to home or office
How will feedback be delivered and reinforced effectively? Some common methods include the following:
- Through trained feedback providers
- In conjunction with training
- In conjunction with individual coaching
- Linked with performance appraisal
- Time 1, Time 2 cycles to track and illustrate improvements
- Through individual coaching sessions
- During training
How well positioned are you to help feedback recipients take action on the resulting data?
- How many employees in your organization will be given feedback?
- Do you plan on repeating the feedback process at a later date to determine progress?
HOGAN ASSESSMENT CERTIFICATION
~August 20-21 in Boston, MA Contact us for details!
~ Onsite Hogan Assessment Certification also available !
CLARK WILSON 360 CERTIFICATION WEBINARS
~ Tuesday April 30, 2019: 8:30 am – 10:30 am EST
~ Tuesday May 21, 2019: 8:30 am – 10:30 am EST
The Task Cycle® is a validated organizing tool that can help leaders know where to focus their development. It is presented as a logical sequence of events, with each phase contributing to achieving the final goal.
Task Cycle® name and image are registered trademarks of The Booth Company.