An Unformed Premise: Critiquing Cole, Article Critique Example

Immediately, in the article’s abstract, Cole et al. establish their thesis and its importance, yet established no definitive methodology for the development of an argument, listing no explicit information regarding research procedures, questions, and objectives (2009). The authors clearly state their intent to establish a correlation between recruiter’s perceptions regarding personality and their employability ranking, as ascertained solely from submitted résumés. The authors further substantiated these arguments with limited support from past research, identifying professional recruiters and graduate students as target groups that will ensure the research will not lack credibility in the academic arena. However, as with the paper as a whole, the thesis and hypotheses were clear, useful, and important but supported by facts which were either too vague or redundantly stated.

The literature fits perfectly in the context of the proposed research in that it described résumés as sources of biographical data, provided expert references considering the ecology model and inferred personality traits gleaned from applicants’ résumés (Mael,1999; Mumford & Stokes, 1992; Caldwell & Burger, 1998). The previous research provided a range of research from a diverse range of journals on business and psychology, limiting undue bias. The authors occasionally seem to forget about the differences between reliability and validity, treating the administration of the Big Five personality tests as an objective measure and not- at least partially- the measure of the changing nature of the human personality. Even the interpretation of such an established test can be questionable or non-representative of a person’s general temperament.

However, Cobb-Clark and Schurer argue that these test results will remain valid for a period of about five years, particularly among working-age adults, and Cobb-Clark and Schurer explain that the Big Five contains some built-in controls which minimize bias (2011). Regarding their understanding of the Big Five personality traits and testing, the authors present a knowledgeable and thorough demeanor, explaining more about the individual traits measured than about their applicability to the thesis at hand. These traits of neuroticism, extraversion, openness to experience, agreeableness, and conscientiousness are defined and explained in accessible terms. The authors’ examination of previous research revealed that objective measures, such as grade point averages and participation, are used by recruiters to determine if qualities such as intelligence, motivation, adaptability, etc., are likely personality traits. This assumption itself is not without merit. Harold (2001) compiles research which indicates that achievement is among the most reliable and valid indicators of possible job proficiency, training and educational success, and counterproductive behaviors. Cole et al.’s exploration of the Big Five tests provides more insights into the reasoning of the authors but provides no understanding of how they seek to study these measures (2009). The information clearly impacts the selection of question items and population samples chosen for the questionnaires, but the validity of the research remains with the previous researchers whom the authors cited.

Despite the potential and dedication which the authors display, stylistic presentation obscures many of the points which they attempt to emphasize. While this saves time, the authors leave their readers preoccupied with making inferences rather than following the development of logical conclusions, creating a disconnect between their carefully-presented facts and the final conclusion reached. They utilize the majority of the cited sources merely to present a fact- rather than explaining the context, relevance, and validity of these studies. This stunted presentation of facts contributes to the scattered nature of Cole et al.’s arguments (2009). This effect could easily have been minimized through the presentation of crucial facts prior to elaborating on their import for the authors’ thesis and through stricter adherence to an accepted methodological form. The article primarily presented qualitative data but occasionally cited statistical examples and referred to foundational case studies in business studies.

Additionally, the majority of the sources cited were published ten years or more before the authors’ article. This would suffice if the paper were to focus on the historical development of employment selections rather than the relevance of prior research to decision-making today. The workplace and its potential employees have significantly changed during that time, possibly altering the crucial dynamics upon which the paper is built. The authors claim that so few new sources are cited due to an extensive oversight of the research literature. However, it seems more likely that their search did not exhaust the various databases or the internet copy available to the mainstream. Ironically, when conducting just such a search personally, an advanced Google search identified this very article as the most relevant and distinctive research on this topic. Interestingly, the authors had conducted a near-identical study five years earlier, which was published in the International Journal of Assessment and Selection (Cole, Feild, Giles, & Harris, 2004).

The authors attempt to use a mixed methods research design, conducting a qualitative literature review and building a questionnaire from which to derive quantitative data. Cole et al. conscientiously select an MBA class to aid them in the preparation of their questionnaires, reiterating their concern about bias within the sample. The authors also select two separate pilot groups with different objectives. Results for both pilot projects were analyzed using the same statistical methods, and convergent validities were tested using the highly-rated NEO Five-Factor Inventory tool, which is similar to the Big Five assessments. From these two evaluative tools, a composite measure is calculated, strengthening the validity of the personal results produced. This is a similar approach to the double-blind testing frequently used in clinical trials- with the exception that such tests require a control to adequately obtain a base line measure. The applicants’ scores on the six-point Likert scales used as part of the Big Five personality tests create a subjective control only for the applicants. In other words, only meager measures establish an understanding of the frequency and severity of recruiters’ faulty assessments of applicants’ personality traits. The authors could refer to Brown and Campion (1994) both for more information and for an example of a sound research design which reliably gauges both qualitative and quantitative measures.

The hypotheses developed by Cole et al (2009), were used to predict the results of the research both in the pilot projects as well as when professional recruiters were engaged, especially in terms of the inferences of the recruiters from the different résumés analyzed, and the reliability and validity of the results obtained. Contact made with Security Human Resource Management (SHRM), facilitated access to the addresses of 5000 recruiters. Only 7 percent of the professional recruiters responded favorably to the solicitation. Generally, the greater the available sample the more representative it is of the diverse human resources typical of a professional recruiter.

The hypotheses were not supported by the results, a result which may be due to the unavailability of experienced researchers who may have exhibited better inferencing skills or greater bias toward applicants. The authors argue that the reasons for the statistical insignificance of the results are the result of applicant–rater interactions as well as the varying informal and unstructured approach used by many recruiters in evaluating resume applicants. In a presentation given by the US Office of Personnel Management, various examples of common assessments are presented, including work sample tests, structured interviews, job knowledge tests, assessment centers, biodata instruments, reference checks, and training and experience point methods, etc. Of these methods listed, the US office reports that the interpretation of biodata fulfills statistical validity requirements—but by a narrow margin (“Assessment Tools Interview”, 2010). Previous authors also support this theory (Bretz et al., 1993; Graves, 1993; Kinicki et al., 1990, as cited by Cole et al., 2009).  Additionally, they posited that to a limited extent, it seems that the recruiters had used certain variables or items to derive the personality attributes of resume applicants, and if some of those inferences were idiosyncratic, they would have been unable to find evidence of reliability and validity, as required in the literature.

Conclusion

Although the information which Cole et al. (2009) presents is relevant, the logical process confuses the reader, sometimes skipping from Point A to Point C if the authors deem Point B to be blatantly obvious in nature. Skipping these crucial steps- however obvious- threatens the validity of the results as the authors show a blatant disregard for methodologies meant to prevent unreliable or confusing journals such as this one. The journal article presents information that is vital and interesting, especially as worldwide recessions continue and the importance of entry-level hires increases. However, the manner in which the information is presented frequently makes it impossible to decipher which information is cited and which is theorized.

The greatest challenge to the validity of this article lay beyond the control of the authors. Questionnaires typically reflect more positive results than are experienced in real situations and cannot adequately address the broad range of tactics, such as the use of position hiring scenarios, which a professional recruiter may utilize. Finally, the recruiter response reflects a general disinterest in the research, a factor which can be assumed to have produced some bias for those who did and did not elect to participate. Statistically speaking, the percentage of returns renders the results negligible—even if the rest of the research is immaculate. We have already established that this certainly is not the case. Thus, the article could be resubmitted if written style is improved, a credible methodology is employed, and the authors can engage more professional recruiters for their research. Until then, the article represents a generalization which is likely true but remains uncertain.

References

“Assessment Tools Overview”. (2010). U.S. Office of Personnel Management. Web. Retrieved from <https://docs.google.com/viewer?a=v&q=cache:g9v7QQ9wStwJ:www.hhs.gov/asa/ohr/new_assessment_overview.pdf+biodata+employment+filetype:pdf&hl=en&gl=us&pid=bl&srcid=ADGEESgjvJTZ18ok0h1ZgEoOyriDhaJiM0SdZ0AjOKBG8PqJARWvuM7B1o8uES8lly6J12IPlXjnyl60JknRAetd5CnxMzJm9fkr2gDedG6IulJGkB2c2lue3CvJnWY-z6PcLIv3ufoR&sig=AHIEtbSJfg8LFSMYvCA1OPZSpeyXUWKNzw>.

Brown, B. & M. Campion. (1994). Biodata Phenomenology: Recruiters’ Perceptions and Use of Biographical Information in Resume Screening. Journal of Applied Psychology, 79(6): 897-908. Print.

Caldwell, D. F., & Burger, J. M. (1998). Personality characteristics of job applicants and success in screening interviews. Personnel Psychology, 51, 119–125.

Cobb-Clark, D. & S. Schurer (2011). The Stability of Big-Five Personality Traits. Institute for the Study of Labor (IZA). Web. Retrieved from <http://ftp.iza.org/dp5943.pdf>.

Cole, M.S., H. Feild, H., Giles, W., & Harris, S. (2004). Job Type and Recruiters’ Inferences of Applicant Personality Drawn from Résumé Biodata: Their Relationships with Hiring Recommendations. International Journal of Selection and Assessment, 12(4). Web. Retrieved from https://docs.google.com/viewer?a=v&q=cache:cthDqa6M_y0J:www.alexandria.unisg.ch/export/dl/michael_cole/38069.pdf+drawing+inferences+résumés+filetype:pdf&hl=en&gl=us&pid=bl&srcid=ADGEESjy6wIn_-6PFbKWrVZazq0ZZSw0_r0MySETppKLX5S08H6MpcKiCyFJ9EQw429pBgtuAWmjoT324VYlDvBYr1Q9vC6gYg_BZvaN6KW7TwRYwG6DZFoZYZK_-4sHOMY4MQDjTpB&sig=AHIEtbRR7AkZxtVZtnbS_YGvKix-GijiVg.

Cole, M.S., Feild, H.S., Giles, W.F & Harris, S.G. (2009). Recruiters’ Inferences of Applicant Personality Based on Résumé Screening: Do Paper People have a Personality? Journal of Business Psychology, 24(5). DOI 10.1007/s10869-008-9086-9.

Harold, C. (2001). Personality, Biodata, and Situational Judgment Tests: A Summary of the Research Literature Comparing Predictive vs. Concurrent Validity. Web. Retrieved from <https://docs.google.com/viewer?a=v&q=cache:YN85GZmizcsJ:mason.gmu.edu/~ion/TechRep.pdf+biodata+employment+filetype:pdf&hl=en&gl=us&pid=bl&srcid=ADGEEShJssHXRpIm0AnGEA9qGwGee_bhgvUyD_aHqMYPdegP44oq4T68h_f3XRNrY7lqK4R5FU-r3JJ16Sxbh-ZlkwNxyHeO8GrcCH68rJeVH-VMg3bzYmCywvvycPkqlZCZq4UcvVoo&sig=AHIEtbRLTYdTbhw3nknHhs3-YAoX_FEvUg>.

Mael, F. A. (1991). A conceptual rationale for the domain and attributes of biodata items. Personnel Psychology, 44, 763–792.

McCrae, R. R., & Costa, P. T. (1992). Discriminant validity of NEOPIR facet scale.