FLY Effectiveness
FLY - Professional
How effective is FLY training?
FLY-Professional Effectiveness Study
Interim Report (December 2023)
Independent observers reported an improvement of 26% in non-cognitive skills exhibited by individuals who took FLY-Professional training, in a longitudinal study.
In 2021 CMI launched a study to determine how ‘effective’ FLY-Professional training really was in the real world.
This study asks independent observers, who work closely with the FLY participant, to score more than 100 questions about the behavior, attitudes and work habits of the FLY participant related to the FLY non-cognitive skills. Questions were asked once at the very start of the training and then again approximately 3 months after completion of the FLY training. The objective of the study is to determine if there is a difference in the actual practice of these five skills by the FLY participant after FLY training.
Analysis of longitudinal data from independent observers shows an average improvement of 26% in each of the 5 skills of the FLY participants. This means that independent observers of FLY-trained professionals saw a noticeable improvement in the behavior of these individuals in the practice of these leadership and competitiveness skills.
The chart below shows ‘before’ and ‘after’ scores (average) for the sample longitudinal group (orange lines show ‘baseline’ Before scores for all FLY participant responders):
In competitive situations, like a swimmer in a race or a technical bid for business, one competitor may consistently outcompete a rival nearly every time even with a small - say 10% - competitive advantage of one kind or another. An observed improvement of 26% in competitiveness (as measured by these five essential constituent skills) should make a substantial difference in the performance and productivity of these FLY participants in any competitive situation.
Study Format
Questions given to the observers are designed to indicate which skill(s), how regularly, and to what extent, the FLY participant exhibits that are related to the five FLY non-cognitive skills: Conscientiousness, Taking Initiative, Perseverance, Innovativeness and Problem Solving. For example, observers rate how much they agree or disagree with statements about the FLY participant (called the ‘Individual’ in the questionnaire) like ‘If I ask the Individual to do something, he/she generally wants to understand why/how it is important.”, or “The Individual mostly considers existing or traditional methods to approach problems.’
‘Before’ and ‘after’ responses are analyzed, normalized, averaged and compared to see if the observers detected a difference in the extended (over 3 months) behavior of the FLY participant, relating to these skills.
Longitudinal Study Results
We looked at longitudinal analysis of approximately 24 participants who had both ‘before’ and ‘after’ evaluations to determine changes in the observed practice of these skills. Below is a summary of the results, which are mapped in the chart above:
Skill | Before | After | Change | % Change |
|---|---|---|---|---|
Conscientiousness | 45.52 | 55.81 | 10.29 | 23% |
Taking Initiative | 43.52 | 56.42 | 12.90 | 30% |
Perseverance | 49.48 | 63.98 | 14.50 | 29% |
Innovativeness | 37.67 | 47.58 | 9.91 | 26% |
Problem Solving | 44.49 | 53.80 | 9.31 | 21% |
Average | 44.14 | 55.52 | 11.38 | 26% |
These results show an improvement in the practice of these five non-cognitive skills in the study participants: between 21% for Problem Solving and 30% for Taking Initiative, with an average improvement of 26%, as measured 3 months after completion of training. Changes in the individual skills are also in a close neighborhood of each other, indicating that the training had approximately the same effect on all the skills, or was equally effective.
The data also show a 51% jump in the average ‘after’ skill score over the ‘baseline’ (before) score, although this is not a longitudinal comparison, and may be due to differences between the ‘before’ and ‘after’ participant groups, or other factors.
‘Baseline’ Scoring
We analyzed results from approximately 215 ‘before’ questionnaires to the observers to establish a ‘baseline’ score in each of the five FLY skills. Baseline scores indicate the level of pre-existing skills, at least among those professionals who signed up for FLY training. Baseline scores in the general population of professionals may be lower due to many either being unaware of the importance of non-cognitive skills or not valuing them enough or not taking the initiative to sign up for training, etc.
Below are the ‘baseline’ scores:
Skill | Baseline Score |
|---|---|
Conscientiousness | 40.07 |
Taking Initiative | 34.34 |
Perseverance | 42.58 |
Innovativeness | 31.48 |
Problem Solving | 35.86 |
Average | 36.87 |
(Maximum theoretical score is 116)
These scores show a consistency (between 31 and 43) that may be expected from questions that are similar in nature although they ask about different behaviors.
These are interim results using a longitudinal sample size of 24. The study is continuing and as more data becomes available, we will publish updated results on our website at www.competitivenessmindset.org under Research & Insights.
Limitations:
The sample size for the longitudinal study was 24 participants. More data is being gathered, which would strengthen findings. These interim results were released after reaching a sample size (20-30) considered sufficient for qualitative, interview-based studies. See ScienceDirect.com, GreenBrook.org, InterQ-Research.com and others.
Statistical significance tests have not yet been carried out on the data for these interim results.