It is now well-accepted that the learning and development process lies at the heart of a nation’s competitiveness. According to both the O.E.C.D. and a leading Human Resource Management (H.R.M.) guru, Professor Dave Ulrich, learning, development and human capital is ‘the DNA of competitiveness’. That is, our national competitiveness stands on a plinth of strong investment in human resources, with learning, development and knowledge recognised as the crucial competitiveness levers. At organisation level, the Irish Business and Employers Confederation (I.B.E.C.) claims that ‘high performing organisations’ recognise the need to use training and development practices to enhance their competitive advantage – with almost 4 per cent of payroll costs allocated to such practices. Related thereto, a recent Accenture survey found that 71 per cent of employers in Ireland have training programmes in place.Whilst at national level, over €415m. has been allocated for 2018 to the National Training Fund, for the purpose of raising the skills of those in employment, jobseekers and to facilitate lifelong learning. The merit of investing in training and development is acknowledged for its beneficial impact on productivity, morale and motivation, attracting and retaining staff, nurturing tomorrow’s leaders and creating a learning culture. Hence, the Management Development Council has no hesitation in promoting the training and development agenda, and has identified significant potential rewards that an improvement in average management performance might yield, noting a McKinsey & Co. analysis that the impact on Gross Value Added in the Irish manufacturing sector could amount to between €500m. and €2.5bn.

Given this backdrop and the increasing preoccupation with assessing the impact of H.R.M.s contribution to the business, the evaluation of investment in training and development is a topical theme. That is, it is now widely acknowledged that it is important to provide evidence of the impact or value added of the investment in this area. Without the evidence, the financial value of such investment cannot be demonstrated and senior management may feel that there is no good reason to continue with this investment. Whilst the training and development process consists of five related components – from needs analysis to design, validation, implementation and evaluation - this article addresses the all-important latter component of the process – the evaluation of the training intervention.

Three Dimensions of Training Process

In summary, there are basically three dimensions of the training process that are measured. Firstly, there’s the participant’s reaction to the training intervention, which normally consists of a series of rating scales, designed to score or measure the trainee’s satisfaction with various dimensions of the programme that they have just completed. Secondly, there’s the participants’ learning from the programme, which may be assessed via a skills or knowledge exercise/test. Thirdly, there’s the impact on ‘on-the-job’ behaviour, as measured via assessments of the trainees’ post-training performance on the job. In reality, various surveys serve to confirm that whilst it is common to assess the reactions, it is quite rare for employers to assess the learning and very rare for them to assess the results or behavioural impact of training and development interventions.

However, given the (often sizeable) scale of investment (e.g. time, resources, fees) in training, it is chastening to note that the most common assessment (i.e. of reactions) is not considered to be as insightful or valuable as the other measurement options. That is, an assessment of how much people liked the programme (i.e. their reaction) offers little insight as to what the participants learned or the behavioural impact on the job. Furthermore, it offers nothing in the way of concrete results – which can be presented to top management, in the form of a return-on-investment (R.O.I.) calculation. Of course, measuring the costs and benefits of training in a manner that enables a R.O.I. calculation is not easily done.

The absence of such rigorous evaluations can also result in wasteful training, that consumes sizeable chunks of the organisation’s financial and human resources. Furthermore, the benefits of any appropriate assessment may materialise in a variety of beneficial ways, such as improving the employer’s position as an ‘employer of choice’, via enhanced employee contributions on the job and the attainment of the organisation’s objectives through advanced skills development or as a result of employee upskilling that enables one to supersede competitors and by enabling the employer to  meet her/his legal obligations (e.g. in health and safety, bullying and harassment, interviewing), ultimately serving to minimise costs and insurance premia whilst maximising best practices.

Evaluation Process

This evaluation process can be quite straightforward and it is often enabled via information technology. For example, the participants’ reactions are often assessed via the post-course ‘happy sheets’ or questionnaires. These are occasionally e-mailed to participants post-course, when they have had an opportunity to reflect on the course’s value/impact. The participants’ learning can be measured via an exercise or test of their knowledge/skill. Whilst the behavioural change may be assessed via feedback from the participants’ staff and/or manager(s) and may be elicited by an appropriate performance management system with a 360 degree or upward feedback dimension.

In an effort to assess the training’s R.O.I., a rigorous ‘scientific’ or ‘quantitative’ approach is recommended by many experts. This entails the development of evaluation techniques that try to focus on objective measurements, whereby the costs and benefits associated with the intervention can be appropriately assessed. For example, both ‘gurus’ in this area - Donald Kirkpatrick and Anthony Hamblin – strongly recommend a tiered evaluation approach, that spans the trainees’ ‘reaction’ to the ‘impact’ of their exposure to the training to the quantification of the economic benefits of the training. This latter assessment or tier tends to include considerations such as the resultant savings in cost and time, improved work habits, superior skills, more initiative, via new ideas and an improved work climate (e.g. as reflected in reduced turnover, grievances and increased commitment and job satisfaction).

Relevant measures of assessment

Relevant measures identified by H.R.M. academics Julie Beardwell and Amanda Thompson in respect of any such assessment include:

• Measures of training activity – How much training/development took place? What is the percentage of payroll allocated? What is the level of expenditure per employee per annum? What is the average number of training days per employee and per manager per annum etc.?
• Measures of training results – What is the percentage of positive course ratings? What is the percentage gain in learning per-course, based on the difference between pre- and post- course results? What is the average percentage of improvement in on-the-job performance after training? What is the average percentage of improvement in profits per employee per annum? What is the training cost per participant per hour and the cost savings as a ratio of training expenses?
• Measures of training efficiency – What is the training cost per delegate hour?

On the same theme, they recommend additional approaches to the assessment of the persuasive R.O.I. ratio. These entail:

• Refining the end of training course ‘happy sheets’ to inquire as to: ‘What will you do (and do differently) as a result of this training?’ and ‘How exactly will each component of this course help you to do a better job?’
• For the purpose of assessing the enduring impact of the training intervention, following up with line managers and the course participants at a later date, asking for their feedback on the course components that they have actually used (and not used) arising from the intervention.
• E-mailing a sample of employees or course participants, asking them about ‘critical incidents’ that they have encountered on the job and what skill-sets they used to deal with them and where they acquired those skills (without prompting that the training course might be the source).
• Interviewing participants, their managers and other significant stakeholders before and after the training intervention, to identify the training/skill gaps or requirements and to gauge the actual impact or value of the intervention.
• Asking participants to complete pre- and post-event evaluations.

However, some specialists suggest that even this assessment approach may be too narrow, as it neglects the all-important longer-term consequences and context of learning and development and fails to address the key questions pertaining to the ‘why’ and ‘how’ training works (e.g. the context, support, methods used). That is, whilst it is accepted that learning and development should show a business benefit, it is also important to appreciate that this should be a considered or ‘staged process’. Hence, the assessment also needs to focus on such matters as the quality of the programmes and their standard of delivery, the organisational culture and the extent to which this enables the application and transfer of the learning to the workplace and the identification of measures in the form of specific and relevant metrics that ought to be identified at the very outset of the intervention.

On this subject, Beardwell and Thompson allude to the metaphor of ‘marriage’ to explain that most training evaluations only consider the ‘wedding’ (i.e. the training event). However, what is really consequential or important is the ‘whole marriage experience’ – that is, the training and all that comes afterwards – the entire training to organisational performance impact.

Factors affecting learning

In support of this perspective, H.R.M. authors John Bratton and Jeff Gold emphasise the importance of diagnosing the factors that affect or prevent the transfer of learning in an organisation. These may include:

• The motivational factors relating to one’s expectations about the application of the new skills. For example, what are the trainees’ perceptions about the value and potential impact of the new information/skills? What is the level of commitment of the learners to use their skills and their belief or conviction in their ability to use them?
• Environmental elements such as peer and management support or sanctions. For example, is the learning encouraged and reinforced by management and built into the goal setting process? What is the likely reaction of co-workers (e.g. as reflected in the cynical or realistic observation by one co-worker to others: ‘He was on a training course yesterday. Don’t worry, he’ll be okay again in a few days.’)
• The ability or scope to apply the new skills and the energy and workload of learners, together with the way in which the training is designed to link to work performance. For example, is there scope on the job that allows the new skills and/or learning to be applied and is the trainer’s style of delivery appropriately practical and engaging?

Hence, Bratton and Gold also extol the merits of a more holistic or comprehensive approach to training evaluation. They recommend an assessment that pinpoints what may be – or more importantly, may not be – occurring, to allow training/development investments to impact on the bottom line. Accordingly, a thorough listing of the evaluation metrics or criteria is necessitated in advance of the intervention, allowing them to be integrated into the programme and the subsequent assessment. This approach is (also) designed to enable parties to identify the barriers to the use of learning and for consideration to be given as to how they can be removed. Ultimately, on this basis the evaluation process helps to set the scene for an enhanced organisational learning climate and is central to an improved training/development impact.

Avoiding assessing training interventions

Of course, as successive surveys reveal, the reality is that many practitioners avoid appropriately assessing training/development interventions. This reluctance may be attributed to the fact that it’s a time consuming and costly process, together with the fact that the benefits of such interventions may be complex to measure, have in-house political implications and complications and are often of little interest to managers preoccupied with more immediate matters. However, the consensus amongst the experts is that the assessment process should go beyond the post-course ‘happy sheets’. That is, it should extend to considerations about the organisation’s culture as it impinges on the training investment, whilst also assessing the contribution of interventions to the entity’s strategic objectives and their impact on customer satisfaction, staff attitudes, morale and motivation, cost-benefits, error rates, wastage etc.

Of course, any comprehensive evaluation recognises that it cannot be narrowly confined to an immediate assessment of the training intervention. That is, training and/or development is an ongoing process that spans one’s life – both on and off the job (i.e. the ‘every day is a school day’ maxim). So, today’s training intervention may well have implications for the development of one’s capabilities and skill-set in respect of future roles in the organisation. As a result, it is closely related to such consequential concerns as management development and talent management, with definite and consequential implications for succession planning. That is, appropriately evaluated training and development interventions are central to the DNA of your organisation’s competitiveness.