The effectiveness of various selection tests in predicting performance has been the subject of extensive research.   Regrettably, however, the conclusions reached are far from unanimous.  Given the growing use of such tests in Ireland, especially for filling managerial posts, this must be a matter of some concern. Hence, testing staff for selection purposes is a process to be treated with care.

Is It A Reliable And Valid Test?

A reliable test is one that enables consistent measurements, at different times and in different circumstances with different subjects e.g. a ruler or a weighing scale. A valid test measures attributes or predictors that relate to job success. The analysis of test scores against performance evaluations is one means of assessing validity. However, a review of the validity of psychometric tests, undertaken by the British Psychological Society, discovered that none could be classed as ‘good’ or ‘excellent’. The message here for practitioners is loud and clear:  ensure that the tests you’re using measure what they purport to measure and that such measures are valid or directly relevant to the prediction of successful job performance.

Beyond the investment risk associated with such tests, there is also a real danger of being legally challenged for the use of biased instruments bearing little or no relationship to job performance (e.g. writing tests for production operative candidates, whose first language is not English).  This has long been a feature of the American recruitment and selection landscape, whilst a number of British employers have also had to deal with such legal scenarios, including British Rail, the Coventry Healthcare Trust and the London Borough of Brent.

Assessment Centres

Assessment centres have the highest predictive validity of all selection techniques.  However, they are an amalgam of tests rather than a test in their own right.  That is, they entail the use of a combination of tests that are designed to help recruiters build a comprehensive picture of candidates. These centres are often used in graduate recruitment, partly because the investment in graduate trainees can be high.

The logic to assessment centres is that the more tests one runs, the more information that is elicited, so the more informed the final decision should be.  It is common for assessment centres to last a full day, whilst some centres can run over a number of days.  The format usually comprises a combination of group work, one or more structured interviews, psychological and personality testing, work simulations and exercises, and perhaps even a ‘trial by sherry’ session.  At the close of the centre, the assessors debrief and evaluate the candidates’ performance using the results from the various assessment exercises.

To get the best out of an assessment centre it is important to ensure that the candidate or person specification is clear, that the different techniques are matched appropriately to the relevant job competencies or criteria and that there is an effective evaluation process for bringing the results of the various techniques into the overall decision-making process.

The most common failings associated with such Centres are:

• Using untrained assessors (e.g. recruiters calling in line managers to help with the workload).

• Failing to link the various tests to the competencies or criteria associated with effective performance on the job.

• The absence of any validation or monitoring of the ongoing effectiveness of the centre.

Of course, where the demand for such an expensive selection device is minimal, or the organisation’s resources limited, the alternative is to use structured or semi-structured interviews and psychometric tests.  These options are adjudged to be almost as good at predicting performance and, if used properly, will probably be far cheaper and certainly more effective than a poorly designed and administered assessment centre.

Work Simulation and Ability Tests

Work simulations involve the design and administration of tests that closely match the tasks involved in the vacant position(s).  Likewise, ability tests of achievement are designed to determine what the candidate already knows or can do, relative to the requirements of the job (e.g. foreign language fluency, legal knowledge, driving skills, word processor knowledge, speed and accuracy).  Once again, the crucial first step is to analyse the job by identifying the key tasks and the skills, qualities or competencies needed to perform them to an acceptable standard.  These tests are acknowledged to be fairly robust, as they seek to assess an observable and measurable skill which is directly related to job performance.

The logic to these test types is that the best way of determining the applicants’ ability to do the job is to give them the job to do.  However, it’s not always possible to design a simulation that fully replicates the job in question.  This could entail a major time investment for the assessment of candidates’ capabilities in every aspect of the job. So, the work simulation tests have to select key elements of the job for assessment and then compress the testing process into an appropriate time-frame.

Practical examples in this context would be:

• For a Human Resources Manager post: drafting a response to a line manager’s request for the summary dismissal of an employee.

• For an academic post:  making a presentation on a specific topic within the relevant discipline.

• For a Public Relations post: drafting a press release on a complex and\or potentially explosive issue.

Biodata Tests

The biodata test dates back to the 1920s and has been impressively validated in predicting a number of work-related factors, including performance, absenteeism, job tenure, and income. Despite its respectable level of predictive validity, it is not widely used.  The design of the biodata test entails asking a group of employees to provide information via a wide-ranging questionnaire.  Having gathered the data one then looks for correlations in the answers that differentiate between the good, medium and poor performers.  The reason for the differences won’t always be apparent or understood, but they do provide the basis upon which to shortlist applicants (who complete the questionnaire enabling such differentiation).  In effect, it’s often coincidence that determines and predicts suitability, rather than any understandable criteria. This test type has been shown to be a valid and reliable means predicting future performance, based on an applicant’s past performance.

Trial By Sherry And Sausage Roll

Trial by sherry and sausage roll selection tests are what the recruitment professionals label ‘introductory dinner’ or ‘meet-the-management sessions’.  They are usually convened the evening before an ‘assessment centre’ day and normally consist of a meal, a drinks’ party or some other such social event.

The main objective of this format is to see what the candidates are like out of the formal assessment environment.  In principle it’s a good idea.  If trainee accountants or solicitors are to find themselves in similar settings with the employer’s clients, then it’s a perfectly legitimate and relevant exercise.  Furthermore, given that some candidates will be more anxious than usual in the formal assessment setting, one might get closer to the real person when they are not being asked to do anything more demanding than eat a meal, make light conversation and ‘be themselves’. However, this approach is based upon some suspect assumptions, such as:

• Candidates genuinely believe they are not being observed and that this will not affect the judgements made about/against them.

• Candidates don’t feel the need to ‘put on a show’ for their assessors because it’s an informal environment without a specific brief.

However, this is often equivalent to a group-based interview, albeit without a clear marking script, standardised process or clear data collection process. However, if the job involves elements of hospitality and socialising, it may be appropriate, though the veneer of informality make it seem more useful than it is.

Ability Tests Of Aptitude

Ability tests of aptitude are designed to predict a candidate’s inherent ability to meet the job requirements.  That is, they measure the applicant’s potential that can be developed to the required standards with training and time on the job.  Such tests measure an occupational aptitude - such as computer programming or sales ability - or be related to a ‘primary mental ability’, such as verbal, mechanical or abstract reasoning, numerical or spatial ability, spelling and language usage or mechanical skills. Yet again, the key issue here is ensuring that whatever aptitude test, or battery of tests, are being used fits the job.  That is, the tests should be reliable and validated, correlating with success on the job.  However, some employers have tended to rely on generic, off the shelf psychometric measures that bear no correlation to job success.

Personality Tests

Personality tests proceed on the assumption that some personalities are more suitable for certain jobs than others. Consequently, an applicant’s personality measurements should enable the employer to predict the broad patterns of behaviour that they would display on the job in the future.  This is important, as personality traits do contribute to success in some jobs and to failure in others.   For example, a shy or introverted applicant is unlikely to be an ideal Public Relations Officer or salesperson.   Over time, a number of different tests have been developed to produce profiles of human traits, which are then used for purposes such as the prediction of suitability for a specific job.  Naturally, the promoters of these tests advance strong claims in support of their product.  Of course, it does stand to reason that if certain personality traits are shown to be essential requirements for success in specific occupations, then with reliable and valid tests designed to assess these traits, the tests can be a valuable component of the selection process.

The two main types of personality test that have been devised are:

Self-Reporting Inventories:  With this format, the applicant is asked to complete a questionnaire, normally in respect of their views about specific subjects and scenarios.  Well established examples of these ‘personality inventories’ are those designed by Cattel (U.S.A.) and Eysenck (U.K.), focusing particularly on levels of introversion and extroversion.

Projective Tests:  In these tests candidates are observed by the selectors over a period of time as they perform a variety of tasks as a team, sometimes with and sometimes without an appointed leader.  Tests of this kind first became established during World War 2, originating in the UK with the War Office Selection Boards. They and are now used by recruiters to assist with the selection of potential leaders, as they can reveal applicant traits and skills in managing or co-operating with others in the performance of actual tasks.  They can also provide useful insights into candidates’ behaviour as group members. It is worth noting that in recent years significant advances have been made to enhance the predictive dimension or validity of the ‘Big Five’ traits (i.e. extroversion, emotional stability, agreeableness, conscientiousness and openness to experience or culture). However, there is no guarantee that the test results translate to the workplace.  Furthermore, the results may represent behaviour measured by the subjective interpretations of human observers in artificial circumstances.   Hence, they’re open to question in terms of their reliability and validity.

Conclusion

An effective selection test will enable the appointment of the most suitable candidate to the job. To ensure that legitimate and successful selection decisions are made, it also helps to garner information vis-à-vis the person specification via a range of techniques (e.g. interviews, tests, references).  Selection tests have become an important part of this process and are adjudged to be cost-effective where the cost of making the wrong decision is a concern. Whilst the evidence on how well personality questionnaires or work sample tests predict job performance is mixed, tests of cognitive ability can be good predictors of job performance, especially for those roles necessitating complex thinking, whilst psychometric tests enable the assessment of individual differences (e.g. in ability, aptitude, personality).

According to the Chartered Institute of Personnel and Development (C.I.P.D.), prior to using a test, one should:

• Ensure that those involved in selecting and administering the tests are competent to do so.
• Consider whether it is appropriate to use a test at all (will it provide additional relevant information?).
• Ensure that the tests are relevant to the role’s person specification.
• Decide how the results will be used and what weight will be given to them
• Ensure that there are no potential equal opportunity issues (i.e. that tests don’t discriminate directly or indirectly against any applicant-type(s)).

Given that the selection process is central to the achievement of the organisation’s goals - and that the only thing the interview can confidently predict is how one will perform at an interview – it looks like selection tests will continue to play a key role in the selection of key staff.

========================================

Dr Gerry McMahon is speaking at the following HR events in Dublin:

Ireland's HR Symposium - 7th March 2019, Crowne Plaza Hotel, Blanchardstown, Dublin

Dignity at Work - How to Respond to Bullying and Harassment - 20th March 2019, Radisson Blu Hotel, Dublin Airport

Successful Selection Interviewing - 9th May 2019, Radisson Blu Hotel, Dublin Airport