Apollo Personality Test Research in Asia
This research study was not part of the original University of Queensland Research. It does however form part of the Asian Personality at Work Research Project and was conducted in association with the Apollonean Institute and Professor Richard Hicks at Bond University, Queensland, Australia.
Outline of the study
In October 2007, PsyAsia International in Hong Kong and the Apollonean Institute in Australia embarked on a research study with students from the Hong Kong Institute for Vocational Education (HKIVE). A traditional (and simplified) Chinese version of the Apollo Profile already existed by way of the publisher producing it for a large client job some time ago. With a plan to develop the Apollo market in Asia, all parties were interested in conducting a reliability and validity study on Hong Kong Chinese people. The project was headed in Hong Kong by Dr. Graham Tyler who has a PhD based on psychometric assessment and validating tools for predicting performance at work in Asia. Mr. Jim Bowden, publisher of the Apollo Profile headed the project from Australia with analytical and statistical support from Professor Richard Hicks at Bond University Queensland.
Phase One of the research required participants to complete an online version of the Traditional Chinese version of Apollo. One month later, Phase Two of the study required the same individuals to complete the same version of the Apollo again (to assess test-retest reliability). Analyses were undertaken by Professor Hicks using data from those who undertook both Phases of testing – a total of 262 Hong Kong Chinese people.
The final phase of the research will compare the student’s personality profiles with their end of year exam result (GPA).
(text reproduced from Professor Hick’s report with minimal editing)
The full profile correspondence of total scale scores at times 1 and 2 was very good overall: test-retest pattern of scale scores being an averaged 0.96 (Pearson Product Moment coefficients across the 34 scale scores). The range was .80 to .99 (SD of 0.024). The conclusions drawn using the full Apollo Profile and its combined scales would have been similar at times 1 and 2.
The Apollo Profile is meant to be used as a whole (that is, combinations of scales or profiles) and no one scale should be used in applications in practice. Therefore the overall patterns of scores used in practice with this Chinese sample (to predict suitability) would have yielded similar results at times 1 and 2. So test-retest reliability is present for the profile pattern.
However, it is also usual to identify how well the scales and sub-scales of questionnaires operate if they were to be used ‘on their own’– and test-retest and internal Alpha coefficients are usually used for this purpose. There is much confusion about the most desirable levels for this latter approach (internal Cronbach Alpha coefficients) and this aspect is addressed later.
The test-retest coefficients by scales, for the Chinese language version, showed a range from .53 to .78 (for the 34 main scales taken individually over the 262 student responses at times 1 and 2: mean: 0.65; SD .08). These are on a scale for scale basis satisfactory though somewhat lower than the coefficients obtained in the English-language version, and much lower than the overall pattern result of average 0.96. The implication is that while the scores on individual scales change somewhat when considered alone, when the scales are considered in concert there is, nevertheless, considerable reproducibility.
The internal consistency figures (alpha coefficients) are generally satisfactory but several scales as shown in the tables need further attention. In the Chinese student sample (on the first administration) 17 of the 28 internal alpha coefficients for the main scales were above .60, 7 scales were between .40 and .59 (on first measure) but 4 of the 28 scales fell outside this range. The major scales appeared to work soundly – these were the scales assessing Agreeableness, Conscientiousness, Extraversion, Innovation-Openness, and Well-being-Resilience (‘Stress Resilience’)- all similar to the Big Five; and Teamwork, Power, Remuneration, Security, Self-organisation, Conformity, Analysing- Analytical, Recognition, Intimacy, Collaboration, Responsibility and Goal-setting.
There were several items (14) identified as causing problems for some of the scales in their alpha coefficients in the Chinese sample; that is, their ‘deletion’ would increase the obtained alpha coefficients. It could be that there are translation or cultural difficulties with these items.
The Pearson PM r between the alpha coefficients (4070) &Chinese test-retest 1-2 coeffs was .70. The ‘order’ is much the same in each case. Those scales that are good or bad in terms of internal alpha consistencies are also respectively good or bad on test-retest.
The Pearson PM r between the alpha coefficients (4070) &Chinese time 1 alpha coeffs was .80 (and .72 at Time 2). The ‘order’ is much the same in each case, but suggests a dropping off at time 2.
The Pearson PM r between the alpha coefficients at Chinese times 1 and 2 is a strong .92, suggesting similar patterns of consistency across the two testings.
The PPMr Chinese T1 and the external Test-Retest coefficient was .72; but between Time 2 alphas and the Test-Retest coefficients just .60. The ‘order’ is somewhat different for the time 2 measure- maybe tiredness or rushing applied to time 2? The drop is consistent at T2 with the external ‘4070’ measure with a similar drop-off found. .
The Time 1 and time 2 order of alpha coefficients was very similar (.92) suggesting despite any ‘drop-off effects’ that similar responses overall were being given on the two occasions. This was supported by the average correlation coefficient of the 262 paired patterns/profiles (of 34 scales) being .964. In general, advice given at T1 would have been similar to the advice given at T2, when profiles of scores and not any individual scales are used. It is not good practice of course in any case to base decisions on individual scales alone and no effective practitioner would do this. The patterns of the scales are what matters. (The use of the Suitability scales follows this practice of using the main profiles).
Exploratory factor analysis
An indicative exploratory factor analysis (varimax in this trial) of the 28 scales not including the six sub-scales/facets for the Chinese sample (of only 262) suggested tentatively that there were five main underlying factors in the responses similar to the factors on the ‘western’ / international samples (with Innovation = Openness, Extraversion, Agreeableness and Conscientiousness factors highlighted, but with a Competitive-conforming factor as 5th, rather than a ‘resilience’ or negative neuroticism factor). These differences need further study. The first five factors were :
Factor 1: Extraversion (.68), Teamwork (.68), Directive (.64), Collaboration (.63), (Negative) Security (-.62), Trust (.61), Ambition (.58), Delegating (.52) and Power (.52) (and others above .30).
Factor 2: Conscientiousness (.85), Self-organisation (.72), Loyal (.59), Collaborative (.50) and (negative) Stress resilience (-.54)
Factor 3: Competitive (.82), Conforming-pragmatic (.76), Remuneration (.59) and Power (.54) with Directive and Compromising but negative Teamwork and Agreeableness (.40s)
Factor 4: Ambition (.54), (negative) Agreeable (-.65), Achievement (.54), Decisive (.50), with Proactive, Self-organisation and Responsible (each .44) and Goal-setting (.38)
Factor 5: Innovation (.53), Analysing (.54), Goal-setting (.52) and Achievement (.45) & Independence (.43)
Of course these results are indicative only and are based on the first testing. Further studies will beneeded to confirm the structures.
There are problems with using a student sample, as a means of confirming alpha reliability coefficients for a questionnaire that is aimed mainly at people in the workplace or about to join the workplace. Further while the test-retest coefficients were on the whole reasonable, I have a concern about the integrity of the second sample. There was some evidence that commitment/ interest/ motivation may have fallen off in the second testing. We need further Chinese studies- both of people in the workplace and of Chinese university students.