Wednesday, April 29, 2015

How to Fail (but Still Succeed) in Corporate Gamification: a Real World Experiment

In February, 2015, I ran a low-tech, pilot gamification process to encourage my company's employees to install an analytics application on their and their family members’ handsets. This report describes the initiation and development of the project, its results, feedback received, analysis of what went wrong and right, and takeaways for future gamification processes.

Gamification in Business

In a business environment, gamification is the application of game and/or play mechanics to a non-game process with the aim of achieving a better result than would be expected without these mechanics. The theory is to motivate people to achieve the process’ goal – not by a job well done or a salary – but by providing engaging elements to the process and imaginary awards designed to be collected in parallel with the process’ goals.

In other words: you add something fun DURING the process that hopefully causes people to do the process well, where previously they did not do the process well due to poor motivation. For example, you include thematic elements (like racing cars or animals in combat), iterative goals, real-time feedback and measurements – like a running score – and a social element that may range from encouragement to cooperation to competition to rivalry. And you might bestow a title upon one or more people who complete the process, or who complete it the best, where “complete the process” hopefully doesn’t mean the person who had the most fun or achieved the highest score, but instead means the person who best completed the non-game process for which we were hoping to get better results.

Businesses are filled with processes that could be done better: showing up on time, showing up altogether, fixing bugs, reducing time for handling customer service calls, handling them better, completing projects quicker, or coming up with kick-ass ideas. For some of these processes, introducing gamification to the process leads to (or can lead to) disaster. If you gamify reducing customer service call time, you encourage faster turnover but worse service. If you gamify creativity, you tend to get less creativity (there are many sources for this).

Gamification works best when it is used to improve simple non-thinking or lightly creative repetitive tasks, like attendance, turnover, and fixing bad records in the computer. Even then, different kinds of tasks are better improved by different types of gamification. It takes some experience, knowledge, and skill to marry the right kind of gamification elements, benchmarks, and goals to the right processes. And because people get bored of the same games over time, applying gamification effectively is a continuous process.

The best way to gain experience in something is to do it. The best place to do a business gamification project is in the company in which you’re already working. Without a clear business case or experience that proves the process is going to save/make a lot of money, you need a pilot that is homegrown and small scale. If it succeeds, you find something larger and more costly to tackle.

The Process We Found to Improve

My company made software for mobile devices, typically installed at source by handset manufacturers or service providers. They were working on an Analytics client that could be installed on a device and which would report back various information about the devices, such as installed OS version, crash times, app usage, and so on.

This was not the first time they were working on this client. They had made two earlier versions. To test it, they wanted in installed on as many devices as possible. On each of the previous occasions, they had tried to get fellow employees to install the client; they got 7 to 15 device installations.

My boss wondered if a gamification project would help. (I had introduced the company, and my boss, to gamification a few months earlier.) Maybe we could get to 50 devices? Or even 100? He approached me. I considered for about ten seconds and said, with confidence, that a company of our size (250 people) could get 1,000 devices using gamification: each employee could install it on his or her own device(s) and three family members of close friends for the two month trial period. If the process was designed and handled properly.

My boss was initially taken aback – 1,000 devices? – but he told me to go for it. The company will get the client ready and the kinks worked out as much as possible, and I will design the game.

The Game as it was First Proposed

This was the email I sent as my proposal for the game:
We have 250 employees (approx) and I think that each one can install it on his or her own device and get 3 to 4 close friends or family to do the same. They only have to keep it on their phone for a month or two. The game is not for the general public, since the client is not yet ready for the public.

The game is as follows: Employees will be divided into six teams, each named for a color and an animal (e.g. brown bears, blue dolphins, etc). Teams include Israeli employees paired with employees in a foreign office. The launch will be at a specific date and end at a specific date. Each team will have use of a dedicated internal email list, and a web page will contain the instructions and each day's and the total scores.

Each employee will get some kind of token, like a pin, to wear in their color during the game.

Employees report daily to me their "captures" (I'm calling the game Handset Hunt), and I update the website with the scores (manually, which is a bit of a pain; we don't have the time to develop something better.) Naturally the server registers all clients, so we know how many devices actually sign up.

If the entire company gets to 1,000 devices, we all get breakfast. The best scoring team also gets dessert (even if we don't get to 1,000 devices).

The Thinking

This game involved nearly no resources; this is a small initiative by a low level employee with no budget, or almost no budget. It is not hard for me to tabulate a dozen or fifty emails in a day, and I wasn’t expecting more than that. The daily feedback would give people a metric to see with constant changes.

I thought two weeks was a good time: a longer period would not only drag out the time unnecessarily, since people would sign up at the beginning and end, but not in the middle, but that is would actually hurt participation, because people would just delay installing the client until they forgot. And that we couldn’t sustain the energy past two weeks.

The task would be simple to accomplish for one’s own devices, and not too difficult to persuade family members for a favor for a few months.

Six teams meant teams of 40 or so players each, pairing somewhat contiguous groups with offices around the world. We tried to keep some kind of coherence, and I was hoping that the tokens and the mailing lists would serve as a catalyst for team identity. I thought a tangible token would be a daily reminder to people that there is a game in progress as well as a possible source of team-identification.

The game included cooperation with teams, to instill a sense of identity, and a competition to foster competitiveness. There was no individual leader board, since I wanted everyone to do something, not something to be done in general by whomever could do it best (which is often what happens with a leader board).

The reward was to provide some additional incentive. Generally speaking, a gamified process solely rewards badges, points, bragging rights, etc. But I figured Israelis, all of whom are post-army, needed something real to incentivize them. Israelis are keen to avoid being a “friar”, that is someone who does something that he doesn’t have to do. They are dedicated when they have to do something that they think is important; many are also dedicated to both cooperation and competition.

The Trigger

I wanted the CEO in Israel, and the leaders in each region, to physically gather everyone together to launch the game and its stress its importance, hand out the tokens, and give some encouragement to participate. Either I didn’t make this clear enough or I was overruled by my boss, I can’t recall which; this trigger, or anything like it, never materialized. Instead, the game was launched by emails sent out with the game information.

Here was one of the emails:

Handset Hunt, a Gamified [Our Company} Game

[Our company] would like to test its Analytics client in the field. To do that, we need to have the client installed on as many devices as possible.

We are inviting you to play our game, which we call Handset Hunt. We are aiming for 1,000 client installations in order to perform a complete test. If we succeed, [our company] will take the entire company out for breakfast.

[Our company] employees are on the prowl to capture and tag mobile devices in order to test their Analytics client. Six packs of hunters will compete to tag the most devices.

Game Goals

[Our company] needs to test the reliability and viability of the Analytics client. The test will be performed by collecting information from many devices over the course of an eight week period.

Team goal: Get the Analytics client installed on the most devices by the end of the game.

Company goal: Get the Analytics client installed on at least 1,000 devices by the end of the game.

Teams

Employees have been assigned to one of six teams. Each team has 39 players.

All employees in each office outside of Israel are on a single team together with as many Israeli employees are required to complete the team size.

You are assign to [Color Animal] team.

Game Period

The game starts on February 4 and runs to February 25 at 23:59 IST (three weeks). This is the “game period”.

Rules

1. Install the Analytics client on as many Android devices as you can during the game period.

2. Ask your family and close friends to install the client on their devices.

3. Inform friends and family to keep the client on their device until at least March 15.

4. IMPORTANT: DO NOT POST A GENERAL REQUEST to your friends on social media or publicize this game or the trial in any public forum. The client is not ready for public use and contains proprietary source code that [our company] does not want exposed at this time.

5. Registration: Each day, email to [someone else in the company] the number of new devices onto which you or your friends or family have successfully installed the client. A daily update will be available on the internal Wiki. The URL is [URL].

6. All teams have an email list to be used as desired (encouragement, discussion, etc). You are receiving this email using this list.

7. If you experience any installation problems, mail [someone else].

Privacy

All information collected is aggregated and anonymous: [our company] cannot map collected information to any individual, and the system ID of the device is not mapped to a phone number.

You can find list of collected information in the game portal: [Portal URL]

Rewards

If the company meets its goal of 1,000 devices, all [our company] employees are awarded breakfast (specifics to be determined). All members of the winning team are awarded dessert (specifics to be determined).

Installation

[installation instructions for Android devices went here]

Teams [1]

  • Blue Dolphins
  • Orange Tigers
  • Red Bulls
  • Green Dragons
  • Brown Bears
  • Yellow Lions

Good luck, happy hunting, and have fun!

The Wiki had these instructions, together with a table at the top of the page listing the names of each person on each team, and the current day’s and total scores.

The Problems

Amateur

The immediate question is “Why not bring in a gamification company?” The HQ of one of the world’s leading gamification companies, Gameffective, was a mere 20 minutes from our office and I am friends with its CEO, Gal Rimon. The immediate answer is that there is no business case for gamification at the company; we were trying to build a business case. Spending tens or hundreds of thousands of NIS was not on the table.

So there was no framework, no individual point scores, no badges, avatars, fancy sounds or graphics, player props, levels, etc, etc.

Teams

The teams didn’t. See below.

Reward

No tangible reward is not a bad thing, and a substantial reward can be a good thing, but a small reward can be a disincentive - like leaving a very small tip. So this was probably a mistake. No one was motivated by the reward, though no one was really demotivated by it, either.

Token

The token was shot down during discussion with my boss and others as impractical; it was difficult to do when our employees were scattered around the world. I suggested that each regional manager could buy the tokens for their employees, but this was rejected. I suggested that, at least, each Israeli employee get a colored something, since I thought something tangible would be important. The result was this change:

The Israeli employees will get a colored paper to place on their desk to write down, which are the phone numbers of the people they got to install the client. Employees also report daily to me. At the end, the Israeli employees give me their papers "for corroboration".

In the end, we abandoned this too, since it was hard to find the right paper in the right colors and it didn’t make sense to give papers to the Israeli employees but not to the non-Israeli ones.

Task

An app that sends data about your phone to a company is not something that most people want on their phone, certainly not without very clear information about what the app does and how long it will stay on your phone. Company loyalty goes only so far. My boss agreed to give general ideas about what it did and a link to the portal with summary info. But he didn’t want to tell people how long to leave it on the phone. I wanted to tell people that they could remove it after two months, but he insisted we leave that out of the message.

So people didn’t have a clear idea of what they were installing, or for how long, or what kind of information was being collected. However, we told them that all data would be kept private to the company.

Also, the process was a fairly finite one. You could play once or twice, maybe a few more times than that, but then you were pretty much done.

Period

I wanted to run the trial for only two weeks. In truth, four days would have been right. My boss wanted four weeks, but agreed to cut it down to three weeks.

The trial was set to start sometime in October, but the client wasn’t ready, so it kept being pushed off. We almost started in December, but the foreign offices indicated that they did not want to run the trial over the end-of-year holidays. I found a new job and gave notice on Jan 19. Finally the trial started on Feb 6, scheduled to end after my time at the company was over on Feb 19.

Culture

I considered whether any of the offices would have a cultural issue with my theme selection of colors and animals. Would being on team dolphins offend the Japanese? Would being on the red team offend the Chinese? I didn’t know, but I didn’t follow through. I don’t think there were any problems.

Support

Any gamification project, unless it is a breakout hit with its audience, requires a champion to marshal enthusiasm among the troupes. In addition to the pitch at the kick-off meeting that didn’t happen, there should have been team leaders, daily events, and an enthusiastic guy (me) to go around spurring people to make the initial leap into the game. I had limited motivation: The game was supported by my company less than I had hoped and I was overruled on some design issues. I knew that I hadn’t made the best game, and the design was made poorer due to compromises that were required and some interference. I was leaving the company. On the plus side, I wanted to see the game succeed.

Reporting

Fielding reports was not difficult, and I dutifully put them up on the web site. However, many people forgot to tell me; we had more handsets connecting the server than were reported in the game.

What Happened

My boss and I made some attempts during the game to remind people about it, I posted once or twice on my team list to start a conversation, but to my knowledge, the lists were never used. Participation was about 10 to 30 handset signups a day during the first few days. By the end of the first week, we had around 120 handsets. A few days later we hit 140, and then nothing until I left the company. I called later for the final counts: 157 handsets.

So we “failed” to reach 1,000 handsets. But we succeeded in reaching over 10 times the number of handsets that had ever been reached in previous attempts. The data collected from the handsets, and the experience gained from the people installing them on various live devices, was invaluable to the client’s development. And it cost nothing.

Feedback

When I asked my boss about the experiment, he wrote: “We completed the development for the trail, we got the client installed on a nice number of devices, and we got insights from the trial. We did not promote it enough and did not get to all people.” In a future project, we should “Invest more in order to have the people on board, call the regions to sign up and call in order to share the insights and not trust mails.”

Here are answers to some questions I asked other people in the company about the game:

1. Did you sign up any handsets?

YES: 8

NO: 7

“I'm proud to be part of the team that signed up the most devices and also one of the two people who personally signed up the most devices.”

“Yes, about 4.”

“I had 5 or 6 handsets installed. All others in my family have iPhones.”

“Only my own.”

“Yes, me and my wife.”

“My family only has iPhones.”

“Yes, I installed the application on my device. I talked a bit with other office members, asking them why they did not installed the client. Half of them are using iOS… and for people who use Android, their device is old and with limited memory.”

“Did not sign up any headsets. Maybe I should have...”

“I signed 3 or 4 handsets not including my own, as the app did not work for me.” [for reasons]

2. Did the "game" motivate you at all to do so (or the general needs of the company)?

“The game definitely motivated me to sign up people.”

“No, I would have done this per personal request also, although it would have been nice to win a good breakfast on the company treat.”

“I think if it wasn’t for the game I would probably install it on my phone and maybe my son’s phone, but with the game I went out to find other people’s phones.”

“The game did not motivate me.”

“It is interesting for me to see the report and I am pretty willing to install the application on my device. I am interested in the game is because I am familiar with the product and I believe it is a valuable product.”

“The game slightly motivated me but not enough to make an actual effort.” “The general needs of the company.”

“The needs of the company motivated me much more than the game/teams/wins etc.”

3. What about the game worked for you and what didn't; what might make you more motivated in the future for similar projects?

It was cool to see the client working

“The fact that we could see the actual results and stats is great. I also see a lot of value in being able to use the server as a real demo and strongly believe that we should eat our own dog food, so to speak – so that worked. ”

“It was nice to receive the summary of the information that was gathered from the devices.”

Lack of triggers or didn’t understand the game

“I didn’t look at [the emails] at all.”

“The emails got lost in the general email clutter - maybe a more physical display of the progress such as the 'thermometers" that the old fund raisers used to use would have been a good idea.”

“One thing I need to complain about was the invitation plan. From the description email, I didn’t understand how to count the invited devices. I remember the email said once we have the application installed on a friends device, we need to write email to someone simply tell the number of the device as a score.. It looks very un-serious to me. Since we don’t need to send either IMEI or some solid proof of a device, while just sending the number of new devices… are you expecting me to be serious about the game?” [He went on to say how the lack of a rigid game methodology in counting devices, combined with the competition, made it seem like the game was based on “trust”, which he couldn’t wrap his head around.]

“This game basically sounded like a spam game to me...”

General problem with the activity

“It’s my personal issue with that kind of software.”

“I did not like the point that I need to install some useless app on my friend's phones... Specifically made for spying after usage statistics...”

“It did not work for me that the people who installed the app got no benefit from it – there could be a report showing some value.”

Lack of daily interaction/feedback

“The game didn't have enough presence or awareness in employees day to day... The reports on progress were not updated daily. There were no motivating follow ups during the game.”

“I did not see my device data in the final report. It is quite a pity to me.”

Problems with the teams

“I think that there was a lack in the sense of real competition, because the groups were not homogenous enough… in the sense that I don’t really know who was in my team, and even what is my team’s name. If my team was “product and marketing”, then I would more easily associate with them and encourage them to do more… but I was brown bears or orange tigers or something else, and the team included people from outside my immediate reference group…”

“Splitting for groups didn't really motivated me. Maybe if it was more individual and I could see online how many devices I registered against the others, it would turn the competition more personal and interesting. :)”

Registering should have been automated [something over which I had no control.]

“I believe the activation of the program should have done the report automatically instead of manual report.”

“Upon installation on device, the application should pop up a screen asking the user to choose which team he wants to register to. This is more intuitive.”

Lack of motivating prizes

“The prizes didn't matter much to me but I would have added some interim (real but small) effort recognition prizes, for example, best team of the week, top signer weekly, top signer, etc.”

“A more serious motivation might be some more real gift to the winners.”

“I would have participated more actively if there was a decent prize involved.”

“Some really cool reward or giveaway or activity for everyone on the winning team may cause more motivation. But the main motivation is not about a reward at all.”

Takeaways

If a lackluster attempt at gamification can achieve 50% participation and produce a 1000% increase in participation for an activity that few people really wanted to do, then there is something about corporate gamification that can work for business. A big portion of that, maybe the bulk of it, is simply promoting the idea that there is an activity outside of the normal work requirements that is important to the company, is not difficult to do, and can be done in a particular time frame, so “please do us a favor”.

It may be that 50% participation was achieved only because this was a “new” thing. Perhaps less people will be interested next time because it won’t be new. Perhaps more people will participate: the ones who did this time will do so again, and new ones will if they know more about it and the game is constructed and introduced properly.

Everyone is wildly different about what does or doesn’t motivate them. No one type of challenge suits all people. A game that provides a number of ways to play that hits multiple motivational points is more likely to succeed. Some people wanted clearer, more defined groups, others wanted more individual competition, while others didn’t care about the competition. The prizes mattered to people – not more than doing their civic duty – but this may have been skewed by the fact that we offered a small prize. If we had offered no prize at all (except bragging rights), maybe fewer people would have asked for a better prize. Or maybe not. However, a better prize would have increased participation.

This game was an example of “short burst” gamification. Different rules apply to long term gamification that is built into the daily routines of employees, Long term gamification has high exec buy-in, is taught to every employee, and becomes part of the daily routine. It must be changed regularly and kept fresh to maintain long term interest, but the rules of engagement are different.

In this process' case, a better game, with a dedicated team, a spiffier framework and graphics, a less frightening product, a more limited time frame, maybe a little less compromises on game design, a more serious buy-in from the execs, and daily enthusiastic communication would have been preferred. These would have gone a long way in achieving better results. I still think we could have hit 1,000.

Yehuda

[1] Pictures taken from various CC sources on Google