The Game Outcomes Project, Part 5: What Great Teams Do

This article is the conclusion to a 5-part series:

• Part 1: The Best and the Rest is also available here: (Gamasutra) (BlogSpot) (in Chinese)
• Part 2: Building Effective Teams is available here: (Gamasutra) (BlogSpot) (in Chinese)
• Part 3: Game Development Factors is available here: (Gamasutra) (BlogSpot) (in Chinese)
• Part 4: Crunch Makes Games Worse is available here: (Gamasutra) (BlogSpot) (in Chinese)
• Part 5: What Great Teams Do is available here: (Gamasutra) (Blogspot) (in Chinese)
• For extended notes on our survey methodology, see our Methodology blog page.
• Our raw survey data (minus confidential info) is now available here if you'd like to verify our results or perform your own analysis.

The Game Outcomes Project team includes Paul Tozour, David Wegbreit, Lucien Parsons, Zhenghua “Z” Yang, NDark Teng, Eric Byron, Julianna Pillemer, Ben Weber, and Karen Buro.

The Game Outcomes Project, Part 5: What Great Teams Do

The following is a summary of our top 40 findings in the 2014 Game Outcomes Project.  We’ve listed the factors in order from most to least important below, sorted by the strength of their correlations.

Our study was based on a 120-question survey of 273 developers.  The resulting data led to some surprising conclusions: we discovered enormous cultural differences between teams, and most of the cultural factors we looked at correlated very strongly with project outcomes.  This article explains what differentiates the most successful game development teams from the rest, according to our survey data.

A large part of our survey was based on team effectiveness models defined in the books Leading Teams: Setting the Stage for Great Performances, The Five Dysfunctions of a Team, and 12: The Elements of Great Managing. Our first surprise was that out of the 46 questions we asked related to these models, 39 (85%) were strongly correlated with every single outcome factor in exactly the way the models predicted, and 5 of the remaining 6 questions correlated with all but one or two outcome factors.

So before you do anything else, pick up those books and read them.  This goes double if you’re in any sort of leadership position.

Just because we’re doing game development doesn’t mean the fundamentals of organizational effectiveness are somehow different from other industries, or that the decades of validated management research done elsewhere is magically rendered irrelevant.

We are not a unique and special snowflake.

Bear in mind that this article is an opinion piece – an editorial that extrapolates from our analysis of the Game Outcomes Project data.  It includes our own subjective interpretations of what our correlations and other analyses actually imply, but only where we felt it was justified by our data (or the models that inspired our survey design, where the data agreed strongly with those models).

If you disagree with any of our interpretations, read the articles (Parts 1, 2, 3, & 4) and form your own conclusions.  You can also download the raw data here in case you’d like to double-check our analysis or investigate the data on your own.

The Top 40: What Great Teams Do

1. Great game development teams have a clear, shared vision of the game design and the development plan and an infectious enthusiasm for that vision [1, 2, 3].  Having a viable, compelling, clear, and well-communicated shared vision was more important than any other factor we looked at.  Make absolutely certain that the vision for the final version of the game is clear and well-communicated throughout the team, and that team members share a similar vision of the game throughout development.  Leads in particular need to communicate a consistent shared vision for the game, carefully communicate changes to the design or the development plan, and resolve any conflicts of vision swiftly and professionally.

Great development teams care deeply about the vision for the game [4].  They have an infectious enthusiasm that sharpens their focus.  This enthusiasm is a huge driver of positive outcomes, and a lack of enthusiasm is a clear warning sign of problems that need to be addressed.  It may mean the vision needs work, or it could be a people problem.  Investigate carefully and don't jump to conclusions.

2. Great game development teams carefully manage the risks to the design vision and the development plan [1].

They are very cautious about making changes in development that deviate too far from the vision.  Fundamental design changes in development are a major driver of increased costs and risks, and are often a sign of deeper problems.

When they disagree about game design, they resolve the disagreement swiftly.  They do not ignore it.

If core design elements DO change, great gamedev teams clearly communicate those changes to the team and justify them.

Some believe that a leader’s job is to “hire talented people and get out of the way.”  This is a deeply flawed notion.  A leader must constantly and proactively work to identify and mitigate any potential threats to the project or the team.

3. Members of great game development teams buy into the decisions that are made [1].  If there’s no buy-in, it's a clear warning sign of deeper problems.

4. Great game development teams avoid crunch [1].

Extended overtime seems to actually makes games worse overall.  As far as our data is able to tell, crunch absolutely does NOT make games better.  Our data provides no convincing evidence that any overtime is helpful in any way.  Not even a little bit.  It has negative correlations with outcomes and positive correlations with poor planning, miscommunication, turnover, and a disrespectful working environment, especially when crunch is mandatory instead of voluntary.

But even if you don’t buy our conclusion that crunch itself makes games worse, you should take a good, hard look at all of the other evidence that crunch increases burnout, disengagement, turnover, and project error rates  … along with the extensive evidence in the broader management research showing that it also harms employees’ health, productivity, relationships, morale, engagement, and decision-making ability, while increasing the risk of alcohol abuse.

The very best results in our study came from teams that were focused and cohesive and worked the LEAST amount of overtime.

If you lead a team, try this exercise: ask your team to work no more than 40 hours a week for 3 months, with the specific goal of increasing productivity and focus as much as possible in those 40 hours.  Genuinely work to optimize your team's productivity during normal working hours and see how much more you can do with less.

And even if crunching were effective, it’s pathetic to ask your team to put in more hours per week before you’ve genuinely done everything in your power to maximize productivity in the first 40.

5. Great gamedev teams build an environment where it's safe to take a risk and stick your neck out to say what needs to be said [1].

If team members don't feel safe and comfortable speaking openly, or harbor any worries about political blowback from speaking their minds, you're likely to miss some very important things.  There could be a hole in your boat, but if people are afraid to call attention to it, you might never find out, and the boat could sink.

Don't let the boat sink.  Don’t let holes go unpatched.  Build an environment where everyone can give and receive honest feedback without getting defensive or political, and respect others’ right to do so.

Some very compelling validated management science shows that this kind of "psychological safety" is essential for building high-performing, learning teams.

6. Great gamedev teams do everything they can to minimize turnover [1] and avoid changing the team composition [2] except for growing it when needed.  This includes avoiding disruptive re-organizations as much as possible [3].

7. Great gamedev teams resolve interpersonal conflicts swiftly and professionally [1, 2].

If you have to bring in outsiders to resolve internal conflicts, you have a problem.

That's not to say all conflict is bad.  Respectful, professional disagreement – or “creative conflict” – should be embraced [3].

But confrontations, politics, and disrespect [4] should not.  Foster constructive politics and ensure that teams stay focused on attacking the problem, not the individual.

8. Great gamedev teams have a clearly-defined mission statement and/or set of values, which they genuinely buy into and believe in [1].  This matters FAR more than you might think.

If your team doesn't have a mission or doesn’t genuinely believe in the stated mission, consider pulling the team together and rewriting your mission statement.

9. Great gamedev teams keep the feedback loop going strong [1]. No one should go too long without receiving feedback on their work.

As part of this, they also practice "no-surprises management" [2].  Give IMMEDIATE feedback and ensure that team members always know how well they are doing.  If there are problems, don't ever wait for a meeting or a performance review to bring them up.

10. Great gamedev teams celebrate novel ideas, even if they don't achieve their intended result [1].

All team members need the freedom to fail, especially creative ones.

Team members are more likely to experiment when they can see that the team and the leads have their back.  This experimentation is key to creativity, and it's an absolutely essential part of building a learning, growing team.

The best teams understand that mistakes are opportunities.

At the same time, this needs to be balanced against the need to carefully manage design risk (point #1 above).  Keep the creative experimentation focused on the right areas, especially on what's needed to complete the game or resolve gameplay problems.  Avoid wasteful design thrashing.

11. Great gamedev teams hold each other to high standards for their particular discipline (art, design, engineering, etc) [1].  Embrace respectful collaboration – including code reviews, design reviews, art reviews, etc. – as opportunities for learning.

12. Great gamedev teams build an environment of mutual respect [1].

Some compelling management research shows that employees who feel respected are significantly more engaged.  This engagement has a direct and measurable impact on project outcomes.

Make sure this respect is maintained as professional dialogue even during passionate disagreements.

Make sure that leads and managers set an example by respecting all team members.

Ensure that respectful behavior is rewarded and disrespectful behavior is swiftly discouraged.

Don't keep team members on board who are unwilling or unable to behave respectfully.  They will only poison the well.

13. Great gamedev teams deal with personnel / HR issues on the team swiftly, professionally, and appropriately [1].

14. On great gamedev teams, everyone on the team is committed to making a great game [1].

15. Great gamedev teams empower team members by ensuring that their opinions count [1].

Ensure that everyone is respectfully listened to and has a chance to change your mind – especially if you're the boss.

16. Great gamedev teams estimate task durations as accurately as possible [1].  This can be difficult, but has a significant impact on outcomes.  Re-estimating task durations on a regular basis to maintain the accuracy of the schedule also seems to have clear positive benefits.

17. Great gamedev teams strive to minimize internal politics and foster an environment where political shenanigans are not acceptable [1].

Don’t let an environment of accountability deteriorate into a culture of blame and finger-pointing.  Your team should be focused on making a great game, not one-upmanship or internecine conflict.

Don't let anyone on the team deliberately act in a way that undermines anyone else's efforts [2].

18. Great gamedev teams discuss failures openly [1].  This helps create an environment of psychological safety.

Failed ideas can be the seed of successful ones.  Sometimes a seemingly bad idea can be inches away from a very good one.

When people to keep failed ideas to themselves, it’s a missed opportunity and a possible sign of team dysfunction.  Mistakes are an opportunity for learning that should be shared, and knowledge hoarding comes with a very high organizational cost.

19. Great gamedev teams don't let any team members put their own priorities above the collective goals of the game project [1].

Team members must never put their own ego, their career, their need for recognition, or their own sub-team or discipline ahead of the team.  If any of these happens, it's a sign of a deeper problem.  Resolve it swiftly, and if necessary, remove the team members in question.

Great teams hold one another accountable and call their peers out on actions and behaviors that are counterproductive to the greater good of the team [2].

20. Great gamedev teams value and utilize the unique skills and talents of all team members [1, 2].  Make sure team members' responsibilities and job roles are carefully matched with their particular skills and abilities.

21. Great gamedev teams enlist all studio stakeholders in decisions to make significant changes to the core game design or architecture [1].

22. Great gamedev teams offer ample praise [1].  Don’t hesitate to call out when someone does a task well.

23. Great gamedev teams keep an open-door policy [1].  Everyone on the team should have easy access to senior leadership to raise concerns, offer feedback, or discuss personnel issues.

24. Great gamedev teams ensure that all team members understand clearly what is expected of them [1].

Team members' tasks should be well-defined and clearly specified [2].

It should always be clear what a team member is supposed to do, and who is supposed to be doing what on the project.

25. Great gamedev teams make the organizational structure and membership of the team clear from the outset and carefully communicate any changes to that structure [1].

26. Great gamedev teams ensure that all team members are well-trained in the studio's production methodology [1].  They also make a deliberate effort to continually hone and improve their production techniques throughout the development process.

Having said that, we see no statistically differences between agile, agile using Scrum, or waterfall production techniques [2].  The only production methodology that shows a difference is not having one: our study shows this is disastrous for teams of any significant size. 

There seems to be no universal right or wrong answer here.  Pick the methodology that you think will work best for your team and your project.

27. Great gamedev teams don't let important things go unsaid [1].  They point out the elephant in the room.

And they ensure that team members have the psychological safety they need to point out big problems and offer challenging feedback.

28. Great gamedev teams give team members opportunities to learn, grow, and improve their skill set [1, 2], and they ensure that someone in the organization encourages each team member to develop his or her skills further [3].

Ideally, this should include both on-the-job development and access to external training, coaching, and mentoring.

29. Great gamedev teams ensure that their team's tools (both software and hardware) work well and allow them to be productive [1].  They keep their game engine running smoothly and their tool chain and asset pipeline running smoothly at all times.

30. Great gamedev teams give team members the authority to determine their own tasks on a day-to-day basis [1].  They also ensure that the person responsible for performing a task is involved in determining how much time is allocated to it [2].

31. Great gamedev teams carefully manage technology changes in development [1], especially large ones.  Switching to a new game engine or making deep changes to an existing engine can be very risky, and great teams take extra care to manage those risks.

32. Great game development teams involve the entire team in prioritizing the work to be done for each milestone or sprint [1].

33. Great gamedev teams meet regularly to discuss topics of interest, ask questions, and identify production bottlenecks [1].

34. Great gamedev teams hold team members accountable for meeting their deadlines [1].

At the same time, they DON'T treat deadlines as matters of life and death, and they don't crucify team members for missing a deadline.  Sometimes expectations aren't reasonable, or design features or new technology features don't work out or take much longer than expected.

Avoid sacrificing team cohesion and morale on the altar of individual deadlines.  The former are worth far more in the long run.

35. Great gamedev teams foster an environment of helpfulness [1].  They reward team members for asking for help and offering support to others.  A "sink or swim" environment will guarantee that everyone sinks in the long run.

36. It's a good idea to have some specs or design documents that describe the vision for the game at the outset [1].  Although these can never replace the work of careful, day-to-day design, design documents show a positive correlation with project timeliness and goal achievement.

37. Great gamedev teams genuinely care about one another as human beings [1].  Treating staff like robots is counterproductive and hurts ROI.

Don’t tolerate “brilliant jerks.”  Or any jerks, for that matter.

38. Great gamedev teams use individually-tailored financial incentives [1].

Financial performance incentives have surprisingly little impact, and they ONLY seem to work at all when directly linked to indvidual performance.  Royalties appear to have no impact on outcomes.  Incentives tied to team performance or MetaCritic scores appear similarly useless.  If you're going to offer financial incentives, consider using Pay For Performance (PFP) plans or similar individual performance incentives.

39. Great gamedev teams – especially large ones – conduct code reviews, pair programming, or peer-reviewed code checkins [1].  These showed a positive correlation with schedule timeliness and meeting project goals, especially for larger teams, and there’s significant evidence that they reduce defects and improve a team’s programming skills.

40. Great gamedev teams recognize that even the best-laid plans sometimes require adjustment.  Any predetermined plan grows increasingly out-of-date as a game project evolves.  Most of the best teams in our survey determined the priorities for each new milestone or sprint based on the current status of the project each time [1].

Experience also matters enormously, but you probably already knew that.  What you may not have known is that it’s about as important as #36 on this list – the first 35 factors we listed all showed a stronger correlation with project outcomes than a team’s average level of experience.

Conclusion

Despite the inevitable risks involved in game development, the clearest result of our study is that the lion's share of our destiny is in our own hands.  It comes down to a culture that consciously and deliberately fosters, cultivates, and supports effective teamwork.

We spend enormous amounts of effort optimizing our code and our art assets.  There's no reason we shouldn't spend just as much effort optimizing our teams, and we hope this study has pointed the way toward some of the tools to help with that process.

The Game Outcomes Project team would like to thank the hundreds of current and former game developers who made this study possible through their participation in the survey.  We would also like to thank IGDA Production SIG members Clinton Keith and Chuck Hoover for their assistance with survey design; Kate Edwards, Tristin Hightower, and the IGDA for assistance with promotion; and Christian Nutt and the Gamasutra editorial team for their assistance in promoting the survey.

For further announcements regarding our project, follow us on Twitter at @GameOutcomes