Log in

No account? Create an account
11 October 2016 @ 09:41 pm
Utah Indie Game Jam 2016 Postmortem  

Last weekend was Utah Indie Game Jam 2016 and I was on a team that made a neat VR puzzle adventure escape game:


Continue on for postmortem thoughts after the cut.

Overall this jam was a learning experience for a lot of people. We didn't win any awards, and our team made a bunch of classic mistakes, but I was still happy with the neat experience I helped create.

What follows is a whole bunch of thoughts of things to do better and ways to do initial idea selection with your jam group. Please add your own if you know of others! I plan to stipulate one of these options for all future teams I join, to see how well I think each of them work.

First tasks that need to be done:

1) Decide on primary communication channel that everyone accesses and checks regularly. It can be supplemented by trello or voice but ideally it is a chat room so people can express anything and can check history for things they want to reference later. Emphasize importance of reporting in often so people don't think you've lost interest or quit.

2) Decide on what to make. Brainstorm with a purpose. Be sure everyone is motivated to make it. Keep scope smaller than time requirement so you can leave room for polish and chill out time at the end.

3) Be sure everyone understands the platform and technical requirements, and that whoever is managing the producer role understands who needs what mentoring on what technology. Randomly assigning tasks results in poor end product quality at best, and complete destruction of the entire team's work at worst (e.g. deleting the repository).

4) Be sure each content creator has an assigned coder to integrate their changes. That coder must keep enough bandwidth open to work on that content integration at all times. If they haven't done that kind of content integration before, they need to not commit to any other tasks for that jam, or else you risk losing that content work.

5) Be sure the group plans to stop one or two hours before, so they can go try out presenting the game on the projector and fix the inevitable "need another cable" problem.

6) Be sure to schedule time for integrating work- two coders working on two things separately, even if they test well on their own, once they come together, there are inevitably bugs or other work that needs to be done to make them a single seamless experience.

The brainstorming game jam problem is somewhat unique, since in a commercial environment you have an easy way to tell what to work on- you work on what the person who is paying for the game to be made tells you to work on. Or maybe you work on what the metrics and feedback from the customers tells you to work on. The following lessons may still apply in real life, e.g. situations where you need to work on a demo but you can't talk to the party you are pitching to beforehand.

Our game jam game mostly failed to get any awards due to presentation. A good chunk of your jam time should be dedicated to the presentation if you want it to go over well. Stop work two hours early and do a presentation practice run-through. This will make it obvious what your priority tasks should be for the final stretch, and help your presentation go over even better for the judges.

When brainstorming, be sure to brainstorm with a purpose. If you just have everyone spout ideas, that's great, you get to know each other and you get exposure to interesting concepts and it might inspire other ideas and it lets people get out their creative expression. But it has no purpose except to get to know each other and get people thinking, unless you do something concrete with it. One of the core problems is cutting out ideas in order to decide on a single vision to work on as a group.

It may be useful to list every idea. To do this, have someone force every idea into a written form of some kind. This lets you use brainstorming as an idea bank for people to reference in the future (after the jam). It also enables certain types of group idea selection methods below. NOTE: When listing, combined or inspired ideas get their own entries, and should be clear about how exactly they use the other ideas.

Here are some ideas on how to decide on the core idea/pitch/vision to start work on initially. Each has varying degrees of flexibility, and different weaknesses to consider.

0) No organization. Sometimes this works well, but the end result is usually a reflection of individual talent on the team rather than a group contribution.

The weaknesses include no support for involving passive developers and introverts, possible loss of interest and motivation due to random things getting worked on and random ideas getting selected, not to mention bruised egos, personality conflicts, and even team meltdown and no work getting finished.

1) Before brainstorming starts to get serious, elect a Vision Leader. This person decides on the core vision after evaluating everyone's ideas and reactions. Note that they are NOT necessarily automatically the leader of the group, or the producer, or anything else like that. It makes sense for them to get those hats as well, but I just want to be clear. Other names for Vision Leader include creative director, and product owner.

The weakness here obviously is that the Vision Leader has to be respected/liked by the group, and good with design, and good with communicating their vision, and not take on too many development tasks so as to keep themselves available to answer questions, or else the end product will suffer.

2) Elect a Vision Selector (or better yet, use an unbiased outside third party). When brainstorming, ideas are brought up and depending on their content and presentation they get varied responses from the group. The Vision Selector should be someone who is tasked with simply evaluating the group responses and then choosing the idea/vision that seemed to have the most favorable group response.

The weaknesses are that this is a very subjective selection (akin to audience cheering judging.. yuck) and it has many of the same weaknesses as the Vision Leader idea. This also has the problem of excluding introverts who may not be very readable in their reactions.

3) Select someone as a Rating Secretary. This person just lists every idea that comes out of the brainstorm until time is up or everyone is done submitting ideas. Remember that combined or inspired ideas get their own entries. Then, once the list of ideas is consolidated and written, the list is presented to the group and everyone gives each idea a rating (e.g. 1=dislike/don't want to work on, 10=like/want to work on). The ratings are averaged out and the highest rated idea wins. If multiple tie for highest, eliminate the others and start over but with those ideas as the core of the brainstorm. Get a list and get ratings again. Repeat until you have just one highest rated idea, and go with that. If you're out of time and the group is okay with it, you could use a random method (e.g. coin toss, smash bros battle, dice, etc.) to determine which of the equally highest rated ideas to go with.

This method includes a built-in way to allow introverted developers to be involved, and gives designers feedback on how they are doing pitching their ideas. The weakness here is that it can be time consuming (e.g. for thinkers to decide on their ratings) and can result in hurt feelings or bruised egos since it necessarily passes value judgement on every idea on the list.

4) Use a Pass/Fail metric to find the first acceptable prototype idea and/or a vision leader. For each presented idea, do a blind simultaneous thumbs up/down on whether you want to work on that idea. Ideas come in first come first served and may be brought up again but not twice in a row. The first idea to get all thumbs up wins, and the idea presenter is the Vision Leader.

This is a simple approach and may be a good fit for an easygoing group who might find it fun. Plus, you get started working FAST. The obvious weakness here is that some better idea that comes up later might be missed out on, and it might feel unfair to people who still had ideas to put forward.

5) The Approval Secretary method (also called Approval Voting or Multi-voting) is very similar to Pass/Fail but instead of selecting the first all-thumbs-up idea, the group has an Approval Secretary note all the thumb results and list them. If an idea has more approval than any other, go with that idea. If multiple ideas get full approval, eliminate all but those ideas, then gather new ideas (e.g. a vision that combines the two without going over scope) and get approvals again. Repeat until you have just one that gets full approval. If you're out of time and the group is okay with it, you could use a random method (e.g. coin toss, smash bros battle, dice, etc.) to determine which of the fully approved ideas to go with.

This has the obvious benefit of ensuring everyone on the team approves of the idea (and so is at least somewhat motivated to work on it). It is more time consuming, but may prove simpler, faster, and just as effective as the Rating Secretary method. The blind voting helps keep votes honest and keeps the focus on the quality of the idea rather than any group drama. The weakness here is that you may run into more ties (which may mean bruised egos, drama, and loss of motivation, especially if tie resolution is not explained beforehand).

6) In a Prototype Vote system (sometimes called Experiment system) you have each person with ideas go implement something (non-coders persuade coders to implement for them, or just come back with diagrams/flowcharts), then bring those implementations back to group and vote which one to go with by simple majority. The prototype that wins has its owner become the Vision Leader.

The downside to this is it takes a huge time chunk (1 or 2 hours implementation time) so the others might get restless.

7) CutStorm (also known as Negative Selection) is a method where you have completely free brainstorm first (explain that it is free and open and just to get a sense of people's preferences), then when time is up for that, have a cutstorm (list ideas, clarify the list, and then cut them down, removing ideas that have objections brought up about them, to finally arrive at the one prototype to make. This can be in freeform discussion/debate format, resorting to votes as desired).

The downside to this is that it may result in bruised egos as the cutstorm devolves into a shouting match, and it also excludes introverted developers.

8) A mixed approach, you could use an Approval Secretary to get a list of feasible ideas, and then use a Rating Secretary to select the idea most likely to succeed in front of the judges.

This has disadvantages of being more complicated and slow, plus all the disadvantages in each of the mixed methods.

9) Evaluation model (also sometimes called Checklist, and similar to Kipling/5W1H method and NUF test, and referred to as a gate in the Phase-Gate model) is an idea where the group creates an idea filter together, forming a document that rates ideas with questions like "How well does this idea fit the theme?" and "How well does this idea fit our team skills?" and gives them 1-10 answers, and also asks open-ended questions like "How will this idea win us an award?" to inspire better ideas. Then each idea is checked to see if it makes it through the idea filter checklist/gate/test. Refine the test until only one idea fits.

This approach is slower and still very subjective but may appeal to some groups.

10) Expert approach. This is similar to Vision Selector, where everyone with ideas refines them and pitches them to an Expert, who gives feedback on them and helps the group decide which idea to run with.

This has the same drawbacks as Vision Selector, plus it requires the group to find an available willing Expert.

11) Voting.Use strawpoll.me or other voting software or methods to clarify group opinions, or to outright select the idea to go with.

This has the advantage of requiring no Vision Leader, but may result in problems down the road since the same pitch can result in vastly different implementations.

12) Work on whatever is Most Interesting to You. This approach involves everyone just working on something and using ad hoc collaboration, with hopes that things will just "come together" at the end.
This has the advantage of generally high individual motivation, but of course may end in disaster as the things people work on may not fit together at the end and much work may be wasted.

13) Vision Pitch method is another way to select a Vision Leader. In this method, after some freeform brainstorming, everyone that would like to be the Vision Leader forms an idea, and pitches their idea to the group. Then the group holds an election and votes for the Vision Leader.

This may result in a better, more qualified selection for Vision Leader. This Vision Leader will also have a strong bias for their idea and may not be as open to input, which can be good and bad. One problem with this method is that it may cause bruised egos and loss of motivation for those not selected.

14) The Nominal group technique (similar to something called the Delphi group method) is an idea for a controlled brainstorm which avoids typical problems such as excluding introverts. First everyone writes down ideas privately and quietly on a paper. Then when time is up, a Nominal Secretary is chosen who collects the papers and begins writing them on a whiteboard or makes a list where everyone can see. The "brainstorm" discussion now at this point does not allow any new ideas. New ideas should be written down privately instead. Open discussion is only allowed when trying to clarify an idea, to be sure everyone understands it. This also helps the Nominal Secretary learn which written down ideas are actually duplicates and which are not. After the list is consolidated, then the Nominal Secretary collects new ideas, and the process repeats. Finally, once time is up or all ideas are listed and understood, an anonymous poll is taken to select the final idea. If the poll results in a tie, the Nominal Secretary starts over from the beginning, starting with a new list consisting of only the ideas that got votes. Again the discussion should only involve clarifying the ideas on the list, not creating new ones. New ideas may only be submitted via quietly writing them on a paper.

This method has the obvious advantage of encouraging everyone to participate, which helps ensure the best ideas are heard. This method may be hard for some people who tend to shout out ideas as they have them. The group must agree that anyone who breaks the rule will accept reprimand. This process can be somewhat time consuming and requires a willing secretary and a method of anonymous voting such as using strawpoll.me or gathering folded papers in a box.

15) Pause method. This simple addition to the ad hoc wild and random brainstorm can significantly improve results and help assuage classic problems with a very simple premise. Do a freeform brainstorm for a set amount of time. Then, Pause. Everyone thinks quietly on their own for 1-5 minutes of uninterrupted thought and evaluation of whatever ideas have been spoken or written down so far. Then resume and see if there is a group consensus or just continue brainstorming.

This is simple and quick, but still holds many of the same problems as freeform brainstorming.

Feel free to comment other methods or send me your thoughts if you like. Thanks for reading!

(Anonymous) on October 13th, 2016 10:54 pm (UTC)
To add a little more, here is some interesting content I overheard at the jam.

"Someone who constantly spews ideas wouldn't last long at a game studio." Disagree? Agree?

"That's an interesting one. I see it a lot with (ambitious) newcomers. They typically learn when it is and isn't appropriate to voice those opinions. I prefer any and all ideas in, say, a brainstorm. I do not, however, like hearing ideas that start to derail something that is already finalized, approved, and in production. Now sometimes that feedback could be a diamond in the rough, but not usually... In my experience, sharing TOO much starts to bog down the production process. It can also alienate you from your peers (if you always come across thinking you are right).
It really can go either way...
The more I think about it... the more I feel like that question pretty heavily depends on context." -Zach Woolf (Game Designer at a game studio)