Do Business Simulation Games Work: The Evidence Case Pt 2

Conceptual Understanding in Simulation Games

In the second article in a series drawn from the research work of Ranchhod et al (2014), available here, I will focus on the use of an online simulation environment to develop conceptual understanding of business concepts, predominantly strategic marketing based decisions, all under the watchful gaze of my countryman Hume.

The scholars set an inventory of learning goals which also included experience generation, skills development and affective evaluation each of which will be covered separately in more detail. However, the focus on ‘experience generation’ is particularly noteworthy as the researchers sought to test for a phenomena that is in many ways the raison d’être of business simulation use, namely the ability of participants to experiment with business ideas and use the simulated environment to take risks that cannot be taken in a real business environment. That is almost precisely how the learning goal was described. If confirmed, it offered credible empirical evidence that supports one of the main claims business simulation specialists make in support of their work.

Conceptual understanding was also to be tested and related specifically to cognitive ability (from Bloom) where the emphasis is on “understanding and retention at conceptual, procedural and strategic level”, further disaggregated by the scholars to include aspects such as terminology, concepts, and principles as well as the interdependence between business functions. The crux being that these concepts require application in complex situations which are, crucially, realistic for the participant. In this vein it was noted from the research that the simulation provides a simplified representation of the business world, yet is also an accurate one. The experience is one of a dynamic system which requires analysis, decision making and implementation of strategy over multiple phases. As such, it is a ‘concrete experience’ to use experiential learning terminology. This then means that it allows for a Lewinian learning cycle where concrete experience is followed by reflection and observation and the formation of abstract concepts and generalisations. Essentially, the ability of participants to be able to formulate their own conceptual models and explanations for the learning experience. In this particular study a main hypothesis was that there would be a causal relationship between the environment of the simulated business world and conceptual understanding of business issues.

In sum, the conceptual understanding learning goals were:

• Understand the theoretical foundations of market competition
• Understand the concept of the strategic perspective
• Understand the theoretical foundations of market behaviour
• Understand the concepts and theory of marketing communications
• Understand the concepts and theory of distribution
• Understand the concepts and theory of pricing
• Understand the theoretical foundations of product management
• Understand the theories and models of information management
• Understand the theoretical foundation of business finance

From experience as an educator working with MBA and MSc Management cohorts I would say this list is pretty intimidating. Even taking relatively experienced and well-educated employees into a simulation with an expectation that they will assimilate this type of knowledge is a considerable challenge. How many senior managers or L&D professionals would be thrilled if they could describe their employees’ acumen in such terms? Many I would warrant.

So what did they find?

Since I’ve taken the time to report this study in such detail it may not surprise you to know that the hypotheses were confirmed with moderate to high value in terms of conceptual understanding from a  single simulation experience. The statistical data demonstrated that the experience value generated by the simulation had “a very strong impact on conceptual understanding” and so reinforced the cognitive outcomes from the pedagogical process. Significantly, the authors highlighted the ability of the simulation to engender effective decision making skills in complex and dynamic situations despite the “simplification and abstraction of the simulated environment”.

Herein lies the single most salient finding from my perspective. Simulation games are inevitably judged in relation to the real world and it is the authors who make a tremendous contribution here by describing the world of the managers working in medium and large sized firms. They are characterised as taking decisions following a process of analysis and interpretation of the available data, working with specialists usually without direct contact with the actual products they sell. They may have tremendous technical knowledge of the commercial offer and value proposition, but it is the value of interpersonal communication and the capacity to analyse abstract information in a market context which affords them success. They operate with high flexibility in unexpected competitive situations and are charged with continuously advancing the organisation’s objectives through the balancing of tactical decisions with long-term strategic goals. Phew. According to the authors this is precisely the process which occurs in a well organised and effective simulation experience.

Empirical evidence. Proof.

And whilst this is only one study I would ask that you take a second to think about the research process. This was no short course with a hastily derived set of learning goals scribbled in bullet point. It was a scientifically designed study which specified in detail not only the method but the precise meaning of every learning goal under scrutiny. It is the antithesis of the generic ‘business acumen’ programme.

Next we will hear what the study had to say about skills development and the affective domain.

Do Business Simulations work? The Evidence Case Pt 1

In a previous article I discussed the most common dilemma in business education, that of ‘external validity’ and the need to prove that investment in training (of any type) requires justification on a typically quantitative level. Most usually that it impacts the organisation positively. In this series of articles I’ll present the findings from scientific research published in peer reviewed journals which details the value of business simulation across multiple domains.

A study conducted recently by Ranchod et al. (2014) built on previous simulation research by testing the use of a high-quality computer-based business simulation. In particular, their focus was on three main categories of learning outcomes built on Bloom’s taxonomy with associated learning goals defined as:

i. Cognitive (understanding and retention at conceptual, procedural and strategic level):

• teach students the terminology, concepts and principles of business in general or of a specific discipline;
• help students understand the interdependence between various business functions (marketing, finance, production, sales
• demonstrate the procedural difficulty of applying business concepts in complex realistic situations; knowledge retention.

ii. Behavioural (skill practice and development):

• enable students to implement course concepts, by taking decisions and experiencing the consequences of their actions in an interactive environment;
• improve students’ team work and relational skills;
• generate practical experience in taking and implementing business decisions;
• improve students’ analysis and decision skills.

iii. Affective:

• improve student attitudes towards the discipline;
• enhance students’ motivation and engagement;
• increase students’ satisfaction regarding the learning experience.

In essence this relates to simulation games as experiential learning (discussed in this blog here) where a Lewinian learning cycle is followed. This means:

0. “the interaction with ‘concrete experience’ leads, through ‘reflection and observation’, to the ‘formation of abstract concepts and generalisations’”

In their study of the literature the researchers noted that there was significant evidence from earlier research in each of the three domains and they stated:

• Students can develop a deeper understanding of fundamental business concepts and procedures, as well as of their strategic significance, during their interaction with the generated experiential situation.

• Previous studies report a positive impact of experiential learning methodologies on students’ skill acquisition. Proposing an engaging, dynamic and interactive learning environment, business simulation games put students in a situation of ‘learning-by-doing’, while the realistic representation of the simulated business systems ensure the transferability of the acquired skills in real-life situation

• Besides cognitive and skill-related outcomes, simulation games research indicates the existence of affective outcomes, expressed through increased motivation, positive attitudes towards the simulation game experience, engagement, general satisfaction, and enjoyment

A critical aspect of the research indicated the dynamic nature of learning outcomes where a causal relationship exists between them. This means individuals are able to formulate their own concepts and generalisations having tested the implications of their knowledge in new situations within the simulations – the basis for skills development.

In fact, the article cites evidence of ‘adaptive expertise’ which is distinct from ‘routine experts’ who solve familiar problems quickly and accurately in that adaptive experts can “innovate and create new procedures to adapted to novel problems and situations”.

The next article in this series will look at the first of the learning goal domains: Cognitive and Conceptual Understanding.

Source:  A. Ranchhod et al. / Information Sciences 264 (2014) 75–90

 

Opening Cinematics in Serious Games

The recent publication of research by Procci and colleagues (available here: http://sag.sagepub.com/content/45/1/93.full.pdf) which focussed on the use of Opening Cinematics in serious games development draws attention to the ever-increasing overlap between the entertainment industry and education. The driver of this overlap is, in many respects, technology but it also speaks to the change in the way individuals expect to consume information regardless of purpose. As we become ever more conversant with technology and, in particular, computing devices we expect to be able to access content for learning in the same way we do for day-to-day communication or entertainment. This crossover is significant since it blurs what are pre-conceived notions of appropriate contexts for learning. Where once we had a well defined and concrete concept of a classroom – the quintessential seat of learning – now we have whole new worlds accessible through a touch screen device. The question, of course, is whether or not this actually improves the learning experience and result.

In essence this is what Procci and colleagues set out to determine. Would the inclusion of an Opening Cinematic which is redolent of the type common in video games, where context and narrative are introduced to motivate individuals to play, be effective in a serious game development? Given the high cost of developing opening cinematics the question is largely one of effective use of resource and ROI. The research focussed on three key areas: Situated Learning, Emotional Arousal and Goal Orientation, each of which is supported by a body of research which has found they are important ingredients in motivation and effective learning.

It’s worth exploring each of these in the context of business simulation where I believe they are particularly evident. The crux of situated learning is that individuals learn more effectively when there is a meaningful context and – crucially – where they can apply the learning. In a business simulation setting we are frequently designing to a context which fits with an organisation’s current view of the world, helping them to work through their ‘burning issues’. As an example, I worked recently with an insurance company which is experiencing a period of fantastic success yet they were challenging their leaders to consider the potential flaws in the organisation which may damage them in the mid to long term.

One such issue was diversity, or to be more precise, the lack of it in their organisation. Within the custom designed simulation I used for delivery we had modelled diversity as a function of the people, skills and experience available in the organisation and directly correlated that with opportunities for transformation. Without the courage to build teams which incorporated and cultivated a diverse range of skills and experience the simulated company was unable to capitalise on opportunities which arose in the market. The success of the simulation was in part due to a context that all participants could engage with as relevant to their business, yet viewed through the lens of situated learning it was the ability to actually practice the development of a strategy for diversity which compounded the learning experience.

The theory and research on emotional arousal posits the view that we are hardwired to recall clearly events that have had an emotional impact and that if training can be designed in this way it will be more deeply encoded resulting in more effective learning. In the simulation environment we deliberately create the kind of stressful environments where cortisol can be released which on a neurological level would explain why simulations are frequently cited by participants as the most effective method of business training when compared with other delivery approaches.

Goal orientation theory is situated within the context of motivation with a distinction drawn in the research between those seeking mastery of a subject and those with a performance orientation. Simply put, the individual who is driven to achieve a goal because it denotes high-performance (particularly in relation to others) is distinct from the individual motivated to master the topic. Where individuals are mastery goal orientated they have a tendency to hone their skills and seek out challenges which help them further refine those skills. In other words, the optimal approach for learning. Within simulation worlds we see individuals demonstrate both behaviours but undoubtedly it is those who aim to understand the dynamics of the simulation and to learn from their decision making processes who gain more from the learning event. Those just looking to ‘win’ can be motivated but, in my experience, lack the ability to reflect meaningfully on what underpinned any success that they gained. More importantly, they can find it difficult to draw useful lessons which they can apply in other contexts outwith the simulated world. They remain ‘in-game’, to some degree.

What is fascinating about this research is that Procci and colleagues concluded there was no significant value add in designing extended opening cinematics for serious games. In fact, they state that “resources should be focused on priming mastery-approach goal orientation and increasing the salience of game goals”. The reason? Most serious games users are already committed to the narrative and context, they recognise the importance of the exercise and are more interested in the value of the learning experience itself. In particular, the ability to practice what they expect to learn.

If Robots Can Be Managers What About Avatars?

In an article by Young and Cormier for HBR this month they describe an experiment which found humans would follow instructions from a robot even beyond the point where they wanted to continue with a task (http://blogs.hbr.org/2014/04/can-robots-be-managers-too/). In a twist from the classic Milgram experiment a 53cm tall robot was competing with a 27yr old human to see who could convince participants to stick longer with a mundane task. The results? 86% to 46% for the human in the lab coat

That may seem like a significant difference until you consider the reality of the result. Under controlled conditions people responded to a robot as though it were in a position of authority and control. Nearly half of the participants were compelled to go through four levels of prompting to keep them on task when they had already expressed a desire to leave. That is simply incredible.

What is most exciting is the author’s insistence that the greatest applications for ‘social robots’, such as the one used in the experiment, is not the mundane and repetitive work we know humans dislike intensely  (with obvious implications for productivity) but those which can truly  leverage the power of a technology with perfect memory, internet connectivity and high-powered CPUs for data analysis. As they rightly point out Robots could therefore be the ideal companion on  projects or be an invaluable asset when making decisions at any level of the business operation.

Yet, when a colleague at VirBELA (www.virbela.com) wondered whether an Avatar in a virtual world could have the same impact, well, that really got me thinking. Increasingly, we can envisage organisations moving beyond the use of virtual worlds like VirBELA for training and virtual team development and on to establishing a presence for team collaboration and project management. Imagine then the power of an avatar who is ever present during meetings and can offer instant access to information both internal and external to the company. An avatar who manages schedules and reminds individuals of tasks and upcoming deadlines. An assistant who is never wrong.

Then imagine the value of their defining characteristic: they are not human. In the workplace, it is the impact of personality which is frequently the bottleneck. The politics and trade-offs that exist in any organisation as individuals strive to give meaning to their work and be recognised for their own merits. In many cases, it can be the management of relationships which defines success or failure, regardless of external market conditions, competitors, demand cycles and so forth.

Perhaps social robots offer an exciting glimpse into the future for both the physical and virtual worlds. Where the convergence of BIg Data and technology results in companions who can provide the results of large-scale analytics and data mining within our very teams. That, and making sure you’re on time for your next meeting. A Brave New World indeed.

Simulations and the Cultural Divide

Lately I’ve been engaged in a project where the theme of global leadership is very much to the fore, with a particular focus on the new and emergent set of competencies managers require to succeed in an increasingly connected environment. Teams are now comprised of multiple nationalities and frequently continue to reside in multiple locations, leveraging advancements in technology which facilitates their work as disparate teams.

Leadership in Multiplayer Online Gaming Environments

What can business learn from leadership in gaming?  

An excellent research article from Timothy C. Lisk and colleagues available here presents a compelling picture of leadership development in MMOs (Massively Multiplayer Online Environments). In particular, they have focussed their research on the relatively new and burgeoning field of Distributed Teams, an area which should be of considerable interest to all businesses given the global nature of communication and work practices today. In fact, as I write this in Glasgow, Scotland my colleagues in the small but high-growth company I belong to are variously located in Brazil, Spain, USA, France and India, and our client list is drawn from an even greater number of countries across the globe. We are the very definition of the Distributed Team who could not exist without web-based communication tools and by developing new approaches to organizational development.

Searching for the Active Substance

What is it that makes computer-based simulation work? What is the magic ingredient that turns training into an event – an experience – instead of the slow, miserable death from a thousand slides we’ve all had the misfortune to witness? Whilst many practitioners would pinpoint the word ‘experience’ as the critical factor, the difference between passivity and involvement or engagement, a group of researchers have attempted to answer the question directly and completely.