WelcomeEnglish|中文

Location:Home > Insights > Leadership Article >

Criterion-Related Validation of the TALENTx7 Assessment®2015/9/11Kenneth P. De Meuse, Ph. D
 Sample:
During the winter of 2016-17, the TALENTx7®self-assessment of learning agility was administered to a group of 61 supervisors, managers, and directors at a large food processing plant located in thesoutheastern region of the United States (U.S.). Two of the participants were eliminated from the analysis, because the “accuracy scales” on the TALENTx7®suggested their learning agility scores on the assessment might not be truthful (i.e., they obtained a 1-star or 2-star confidence index rating). In addition, 16 were removed from the analysis since they were hired during the past year and had received no performance or competency ratings at the time of this study.

Of the remaining sample of 43 participants, 25 (58%) were supervisors, 14 (33%) were managers, and 4 (9%) were at the director level. Eight (19%) of the participants were female. Their years of leadership experience ranged from 2 to 24, with a mean of 6.23 years.

Ratings of Leadership Competence Each year, all leaders in this company are evaluated on 12 leadershipcompetencies by their immediate manager. The 12 competencies include:
Adaptability to change
Business and financial acumen
Collaboration/teamwork
Communication
Customer focus
Decision making
Directing and developing others
Drive for results
Innovation
Integrity and moral courage
Problem resolution
Strategic capability
Appendix A describes each competency in greater detail. 

A 5-point scale is employed to rate each of those competencies, ranging from substantially belowexpectations (1), below expectations (2), meets expectations (3), above expectations (4), toconsistently exceeds expectations (5). To simplify the analysis, a composite rating of the 12 competencies was computed. Further, an overall mean composite rating for a three-year periodwas calculated for those participants who were in their leadership position for multiple years.

All rights reserved. 2 Ratings of Learning Agility The psychological construct of learning agility was measured by the TALENTx7 Assessment®. 

This online self-assessment measures a participant’s learning agility on seven different facets, as well as his or her “Overall Learning Agility.” Scores are converted into percentiles to provide individuals their level of learning agility relative to others (De Meuse & Fang, 2017). Table 1 presents the learning agility scores by organizational position level.

As expected, the directors outscored the supervisors and managers on “Overall Learning Agility” and on five of the seven facets. The mean score differences were particularly noteworthy on “Cognitive Perspective,” “Drive to Excel,” and “Self-Insight,” where mean scores for the director-level participants were especially high. Somewhat surprisingly, supervisors generally had higher learning agility scores than managers. For example, the mean score on “Overall Learning Agility” for the supervisory group was M = 52; whereas, it was M = 42 for the managerial group. (As can be seen, it was M = 63 for the director-level group.)

It would seem logical that the importance – and prevalence – of learning agility would increase as one ascends the organizational ladder. Indeed, many researchers have asserted that different competencies and behaviors are required as leaders transition from lower levels to upper levels in management (see Charan, Trotter, & Noel, 2001; Lombardo & Eichinger, 2011).

Since learning agility is helpful during this organizational journey, one would hypothesize directors would possess more of it than managers (observed here) and managers would possess more of it than supervisors (not observed here).

Table 1. Mean Learning Agility Scores by Organizational Position Level Learning Agility Facet Supervisor Manager Director Interpersonal Acumen 57 47 64 Cognitive Perspective 63 44 81 Environmental Mindfulness 49 58 60 Drive to Excel 65 62 84 Self-Insight 42 28 71 Change Alacrity 48 46 45 Feedback Responsiveness 41 37 39 Overall Learning Agility 52 42 63 Note. 

There were 25 supervisors, 14 managers, and 4 directors included in the above analyses.All rights reserved. 3 Relationship between Leadership Competence and Learning Agility Table 2 provides the correlation coefficients between the participants’ composite leadership competency ratings and learning agility scores. To estimate the true relationship between the two sets of scores, coefficients were corrected for unreliability and restriction of range in the competency ratings (see Guilford & Fruchter, 1978). As can be seen, “Overall Learning Agility” had a very strong relationship with leadership competency ratings (r = 0.62, p < .001). 
Moreover, each of the seven specific facets of learning agility measured by the TALENTx7® also correlated significantly with competency ratings, ranging from a low of r = 0.30 (“Change Alacrity”) to a high of r = 0.66 (“Environmental Mindfulness”).

Table 2. Relationship between Overall Leadership Competence and TALENTx7® Scores of Learning Agility  Learning Agility Facet Relationship
r p
 Interpersonal Acumen 0.63 p < .001
 Cognitive Perspective 0.34 p < .05
 Environmental Mindfulness 0.66 p < .001
 Drive to Excel 0.62 p < .001
 Self-Insight 0.40 p < .01
 Change Alacrity 0.30 p < .05
 Feedback Responsiveness 0.45 p < .01
 Overall Learning Agility 0.62 p < .001
 Note. N = 43.
Overall, this analysis reveals a very robust relationship between learning agility as measured by the TALENTx7 Assessment® and leader competency ratings. The finding becomes even more noteworthy given that the ratings were obtained by calculating a composite of 12 leadership competencies over a period of three years. Consequently, the competency scores, which were correlated with the TALENTx7®
learning agility scores, represent an amalgamation of each leader’s perceived competence over multiple years.

A previous study investigating the criterial-related validity of the TALENTx7® also found support for the assessment (De Meuse, 2016). In that study, annual performance ratings – rather than competency ratings – were collected in a large U.S. retail company. Thus, both of these studies applying different criteria and located in dissimilar industries provide strong evidence of the validity of the TALENTx7 Assessment®.